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MESSAGE  FROM  THE  DIRECTOR 


Many  people  are  asking  how  Alberta  students 
are  doing.  There’s  a mixed  answer.  Some  are 
doing  very  well,  according  to  the  provincial 
achievement  tests  written  last  June.  In  two  of 
the  subjects  tested,  Grade  3 Science  and 
Grade  6 Math,  more  students  than  expected 
performed  at  the  standard  of  excellence.  But 
the  test  results  also  show  that  the  majority  of 
students  aren’t  doing  well  enough.  A smaller 
proportion  than  expected  met  the  acceptable 
standard  of  performance. 

During  the  testing,  we  carried  out  a special 
study  comparing  students’  achievement  now 
with  that  of  students  in  1987  and  1983.  The 
results  of  the  study  are  reported  in  Section  8 
of  this  report  and  show  that  students  are 
doing  better  than  in  the  previous  years.  In 
1991,  for  example,  a higher  percentage  of 
Grade  3 Science  students  correctly  answered 
process  skills  questions.  The  study  was  a 
complex  one.  It  gave  us  useful  information 
and  helped  us  make  some  improvements  in 
design  for  future  studies.  We  would  like  to 
express  our  appreciation  to  the  teachers, 
principals,  and  superintendents  who  helped 
us,  and  we  look  forward  to  the  participation  of 
schools  in  future  projects. 

In  this  year’s  report,  our  numbers  express  the 
proportion  of  students  achieving  standards  on 
the  two  parts  of  each  test — knowledge  and 
process  skills.  To  reach  an  acceptable  stand- 
ard, students  had  to  succeed  on  questions  in 
both  areas.  This  is  harder  than  gaining  a 
certain  score  based  on  questions  from  just 


any  part  of  the  test.  We  expect  at  least  80  per 
cent  of  all  students  within  our  defined  popula- 
tion to  achieve  the  acceptable  standard,  and 
we  expect  at  least  10  per  cent  of  all  students 
within  the  same  population  to  achieve  the 
standard  of  excellence. 

This  is  the  last  year  that  we  will  report  scores 
for  the  total  test. 

For  the  first  time,  we  have  formally  sought  the 
views  of  the  public  on  how  students  are  doing. 
A public  advisory  meeting,  attended  by  repre- 
sentatives of  business,  professional,  and 
community  groups,  told  us  that  our  young 
people  should  be  doing  better  than  the  test 
results  showed.  In  the  coming  years,  we  will 
continue  asking  the  public  if  student  perform- 
ance is  good  enough. 

In  the  next  round  of  achievement  testing,  we 
will  collect  information  about  what  students 
know  and  can  do  in  a few  new  performance 
areas: 

• problem-solving  tasks  in  math  and  science 

• listening  and  viewing  skills  in  language  arts 

• participation  skills  in  social  studies 

We  are  looking  forward  to  including  in  next 
year’s  report  information  that  will  tell  us  more 
about  students’  achievement.  Such  informa- 
tion helps  us  to  focus  on  high  expectations  and 
on  ensuring  that  students  do  their  best. 
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SECTION  1 

SUMMARY  OF  ACHIEVEMENT  TEST  RESULTS 


Parents,  educators,  and  the  general  public 
need  to  know  how  well  Alberta  students  are 
achieving  in  relation  to  provincial  curricular 
expectations. 

The  1991  achievement  test  results  help  dem- 
onstrate what  is  possible  for  grades  3,  6,  and 
9 students  to  know  and  to  do  in  Science, 
Mathematics,  and  Social  Studies  respectively. 

“Knowledge  of  what  is  possible  produces  new 
enthusiasm,  raises  sights,  establishes  new 
challenges  and  ultimately  can  improve  per- 
sonal and  societal  performance.” — Learning 
Mathematics/Learning  Science  International 
Assessment  of  Educational  Progress,  Educa- 
tional Testing  Service,  February  1992. 

Guidelines  for  interpreting  the  1991  results 
are  given  in  Appendix  C,  and  Appendix  D 
provides  answers  to  a number  of  questions 


frequently  posed  by  parents.  The  process  of 
developing  the  achievement  tests  is  outlined 
in  Appendix  E. 

This  section  of  the  report  describes  certain 
broad  characteristics  of  the  student  population 
who  wrote  the  achievement  tests  and  provides 
answers  to  the  following  questions: 

•How  many  students  wrote  the  achievement 
test  for  their  grade  and  how  many  students 
did  not  write  the  achievement  test  for  their 
grade? 

• What  percentage  of  the  students  attained 
the  acceptable  standard  according  to  provin- 
cial criteria? 

• What  percentage  of  the  students  attained 
the  standard  of  excellence  according  to 
provincial  criteria? 


For  the  first  time,  results  are  reported  in 
terms  of  three  related  but  different  stand- 
ards: curriculum  standards,  achievement 
standards,  and  assessment  standards. 

Curriculum  standards  are  the  expected 
student  learnings,  sequenced  into  levels  that 
are  stated  for  each  curriculum.  They  include 
specific  statements  of  knowledge,  skills,  and 
attitudes  against  which  student  performance 
is  to  be  judged. 

Achievement  standards,  usually  expressed 
as  percentages,  state  how  many  Alberta  stu- 
dents are  expected  to  reach  a particular  curri- 
culum level  at  a specific  age  or  year  in  school. 
It  is  important  to  point  out  that  this  judg- 
ment is  not  a prediction  of  what  percentage  of 
students  will  actually  achieve  acceptable  or 
excellent  levels  of  performance  but  rather 
specifies  the  percentage  of  students  at  a given 
age  or  year  in  school  who  are  expected  to 


achieve  the  acceptable  or  excellent  levels. 
These  standards  apply  to  school,  jurisdiction, 
and  provincial  achievement. 

Assessment  standards  are  the  scores  to  be 
achieved  by  a student  on  a specific  test  or  part 
of  a test  before  the  performance  of  that  stu- 
dent is  judged  to  be  “acceptable”  or  “excellent” 
relative  to  the  curriculum  standards.  Alberta 
Education  reports  the  results  for  the 
achievement  tests  in  relation  to  an  acceptable 
standard  and  a standard  of  excellence  level 
respectively. 

In  previous  years,  assessment  standards  for 
the  total  test  and  for  individual  components 
were  determined  and  reported.  In  1991,  in  an 
effort  to  provide  more  meaningful  and  useful 
results  to  schools  and  jurisdictions,  achieve- 
ment and  assessment  standards  are  reported 
for  each  major  component  of  the  tests  and 
for  both  components  together. 
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Assessment  and  achievement  standards  for 
the  1991  achievement  tests  were  determined, 
and  subsequently  recommended  to  the  Direc- 
tor of  Student  Evaluation,  by  teams  of  cur- 
riculum specialists,  test  development  special- 
ists, and  experienced  subject  classroom 


teachers  who  applied  standard-setting  proce- 
dures to  the  tasks  under  the  guidance  of  the 
Analytic  Services  Unit  of  the  Student  Evalua- 
tion Branch.  An  outline  of  the  processes 
followed  is  provided  in  Appendix  A. 


STUDENT  POPULATIONS 


In  June  1991,  principals  reported  a total 
population  of  108  334  students  in  the  grades 
tested.  There  were  39  216  students  in 
Grade  3,  36  565  students  in  Grade  6,  and 
32  553  students  in  Grade  9.  Of  that  total 
population,  97  858  wrote  achievement  tests. 
The  remaining  10  476  students  were  ex- 
empted from  writing  or  were  absent  the  day 


the  tests  were  administered,  as  shown  in 
Figure  1-1.  The  tests  were  administered  in 
several  special  forms  in  addition  to  the  regu- 
lar form.  The  information  collected  from  the 
special  forms  was  used  to  investigate  student 
achievement  over  time  and  the  effects  of  the 
language  of  testing. 


Figure  1-1 


STUDENTS  WRITING  ACHIEVEMENT  TESTS 
June  1991 


* The  number  of  students  writing  the  tests  includes  students  in  the  regular  English  program  as  well  as 
Francophone  and  French  Immersion  students  who  wrote  the  achievement  test  in  either  English  or  French. 

Provincial  results  are  based  on  test  scores  achieved  by  students  who  were  in  the  regular  English  program,  which 
is  defined  in  the  first  footnote  for  tables  2-2,  3-3,  and  4-3.  Results  for  Francophone  and  French  Immersion  stu- 
dents are  reported  directly  to  participating  schools  and  jurisdictions  and  are  not  calculated  in  provincial  results. 
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OBSERVATIONS  ON  STUDENT  PARTICIPATION 


The  student  participation  rate  was  highest  in 
Grade  6 (92.5%)  and  lowest  in  Grade  3 
(88.8%)  when  calculated  from  the  numbers 
reported  on  the  Principal’s  Reports  submitted 
after  each  achievement  test  administration. 
However,  the  lower  participation  rate  in 
Grade  3 Science  may  be  because  there  was  no 
French  translation  of  the  test  for  those  stu- 
dents instructed  in  French.  In  terms  of 
absolute  numbers,  more  Grade  3 students 
(34  825)  wrote  their  test  in  Science  than  did 
Grade  6 students  in  Mathematics  (33  814) 
and  Grade  9 students  in  Social  Studies 
(29  219). 


When  participation  rates  are  calculated  from 
the  September  30,  1990,  enrolment  figures, 
however,  another  picture  emerges.  In  Grade  3 
Science,  the  participation  rate  decreases  to 
84.9%;  in  Grade  6 it  falls  to  91.1%;  and  in 
Grade  9,  it  is  reduced  to  82.3%.  In  terms  of 
numbers,  approximately  1 785  Grade  3 stu- 
dents, 549  Grade  6 students,  and  2 959  Grade 
9 students  were  not  accounted  for  administra- 
tively at  testing  time.  Obviously,  administra- 
tive accounting  must  be  enhanced  before  the 
next  round  of  testing. 

Details  of  student  participation  at  each  grade 
level,  including  the  number  of  students  who 
were  absent  or  exempted,  are  presented  in 
sections  2,  3,  and  4 of  this  report. 


RESULTS  IN  RELATION  TO  STANDARDS  FOR  STUDENTS  WHO  WROTE 
THE  ACHIEVEMENT  TESTS 


For  the  1991  results,  the  focus  of  reporting  of 
student  achievement  is  on  the  major  compo- 
nents of  the  test.  Results  are  reported  in 
relation  to  the  standards  established  for 
individual  major  components  as  well  as  for 
both  components. 


Based  on  discussions  with  educators,  test 
development  specialists,  and  curriculum 
specialists,  and  on  our  experience  with  meas- 
uring student  achievement  according  to  the 
expectations  in  the  Program  of  Studies, 
achievement  standards  were  confirmed  as 
follows: 


Acceptable  Standard 

Standard  of  Excellence 

Each  major  component 

85% 

15% 

Both  major  components 

80% 

10% 

Achievement  standards  were  also  confirmed 
for  the  total  test: 

Acceptable  Standard  85% 

Standard  of  Excellence  1 5% 

Standards  reflect  expectations  for  the  regular 
English  program  students  and  not  for  the 


total  population — which,  for  example,  would 
include  special  needs  students. 

Figures  1-2  and  1-3  present  the  percentage  of 
students  who  achieved  the  acceptable  stand- 
ards and  the  standards  of  excellence  based  on 
their  scores  on  both  components  of  the  test. 


-3 


Figure  1-2 


PERCENTAGE  OF  STUDENTS  ACHIEVING  ACCEPTABLE 
STANDARDS  OR  HIGHER*  ON  BOTH  COMPONENTS  OF  THE  TEST 

June  1991 


*Includes  students  achieving  the  standard  of  excellence 
**80%  of  students  were  expected  to  achieve  the  acceptable  standard  or  higher  on  both  components  of  the  test. 


Figure  1-3 


*10%  of  students  were  expected  to  achieve  the  standard  of  excellence  on  both  components  of  the  test. 
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Figures  1-4  and  1-5  present  the  percentage  of  ard  or  the  standard  of  excellence  based  on 
students  who  achieved  the  acceptable  stand-  their  total  test  score. 

Figure  1-4 


100- 

80 

60 

40- 


PERCENTAGE  OF  STUDENTS  ACHIEVING  ACCEPTABLE 
STANDARD  OR  HIGHER*  ON  TOTAL  TEST  SCORE 
June  1991 


78.9 

75.5 

| 78  9 

** 


Science  (Gr.  3)  Mathematics  (Gr.  6)  Social  Studies  (Gr.  9) 
(N  = 33  837)  (N  = 31  930)  (N  = 27  646) 


*Includes  students  achieving  the  standard  of  excellence 
**85%  of  students  were  expected  to  achieve  the  acceptable  standard  or  higher  on  the  total  test. 


Figure  1-5 


*15%  of  students  were  expected  to  achieve  the  standard  of  excellence  on  the  total  test. 
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OBSERVATIONS  ON  GENERAL  RESULTS 


In  Grade  3 Science,  results  based  on  both 
components  of  the  test  revealed  that  the 
percentage  of  students  achieving  standards 
was  lower  than  expected  for  the  acceptable 
standard  but  slightly  higher  than  expected  for 
the  standard  of  excellence . This  pattern  was 
similar  to  that  of  results  based  on  the  total 
test  score,  where  levels  of  performance  were 
lower  than  expected  for  the  acceptable  stand- 
ard but  slightly  higher  than  expected  for  the 
standard  of  excellence . 

Results  for  Grade  6 Mathematics  exhibited  a 
similar  pattern.  However,  a smaller  percent- 
age of  Grade  6 students  than  Grade  3 students 
achieved  the  acceptable  standard  but  a larger 
percentage  of  Grade  6 students  than  Grade  3 


students  achieved  the  standard  of  excellence . 
This  is  true  for  results  based  on  both  compo- 
nents of  the  test  as  well  as  for  results  based 
on  the  total  test  score. 

For  Grade  9 Social  Studies,  results  were 
disappointing  for  both  levels  of  performance. 
The  percentage  of  students  achieving  stand- 
ards was  lower  than  expected  for  the  accept- 
able standard  as  well  as  for  the  standard  of 
excellence. 

In  conclusion,  results  showed  that  students  at 
all  three  grades  need  improvement  to  achieve 
the  acceptable  level  and  Grade  9 students  also 
need  improvement  to  achieve  the  standard  of 
excellence. 
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SECTION  2 
GRADE  3 SCIENCE 


GENERAL  DESCRIPTION 


The  Grade  3 Science  Achievement  Test  con- 
sisted of  50  multiple-choice  questions.  The 
time  allotted  for  writing  the  test  was  25 
minutes  for  each  of  two  sections.  Statistics  for 
the  total  test  and  for  the  components  are 
based  on  the  results  achieved  by  33  837 
students  (30  427  wrote  the  regular  form,  and 
3 410  participated  in  the  achievement-over- 
time study,  as  shown  in  Table  2-2).  This 
section  of  the  report  provides  answers  to  the 
following  questions: 


How  many  Grade  3 students  wrote  each 
form  of  the  test  or  were  absent  and  exempt? 
What  percentage  of  Grade  3 students 
attained  the  acceptable  standard  in 
Science  according  to  provincial  criteria? 
What  percentage  of  Grade  3 students 
attained  the  standard  of  excellence  in 
Science  according  to  provincial  criteria? 
What  did  Grade  3 students  know  and  what 
could  they  do  in  Science? 

What  parts  of  the  Science  curriculum 
caused  Grade  3 students  difficulty? 


SUMMARY  OF  RESULTS 
Results  in  Relation  to  Standards 

Results  show  that  70.1%  of  students  who  wrote  the  test  achieved  the  acceptable  standards  on 
both  components  and  11.0%  achieved  the  standards  of  excellence  on  both  components.  These 
results  were  lower  than  expected  for  the  acceptable  standards  but  slightly  higher  than  expected 
for  the  standards  of  excellence. 

Average  Score 

The  average  total  test  score  was  68.4%,  with  a standard  deviation  of  18.2.  The  average  total 
test  raw  score  was  34.2  marks  out  of  a possible  50,  with  a standard  deviation  of  9.1. 


CONTENT  OF  THE  TEST 


The  Grade  3 Science  Achievement  Test  was 
designed  to  reflect  the  Grade  3 Science  Cur- 
riculum Specifications  (revised  May  1986) 
that  were  developed  from  the  revised  1984 
Science  Program  of  Studies.  The  scope  of  the 
test  was  limited  to  curriculum  objectives  that 
could  be  efficiently  measured  on  a paper  and 
pencil  test.  As  a result,  questions  on  the  test 
were  drawn  from  the  content  of  the  two  major 
components  in  the  core  program: 


• Subject  Matter 

• Process  Skills 

The  subject  matter  component  consisted  of 
questions  associated  with  the  major  themes  of 
science  in  Division  I.  The  process  skills 
component  consisted  of  questions  integrated 
with  subject  matter  and  questions  independ- 
ent of  subject  matter. 
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Questions  on  the  Grade  3 Science  Achievement  Test  measured  student  achievement  in  two 
cognitive  levels: 


• Knowledge — recognize  or  recall  ideas, 
terminology,  facts,  conventions,  methods  of 
inquiry,  principles,  generalizations,  theo- 
ries, and  concepts 


• Comprehension  and  Application — demon- 
strate an  understanding  of  the  concepts  and 
skills,  and  apply  appropriate  methods  and 
ideas  to  a new  situation 


ACHIEVEMENT  TEST  BLUEPRINT 

Table  2-1  presents  the  blueprint  used  to  develop  the  Grade  3 Science  Achievement  Test.  Clas- 
sification of  each  question  by  component,  cognitive  level,  and  subtest  is  indicated  in  the  table. 


Table  2-1 
Grade  3 Science 
Achievement  Test  Blueprint 
June  1991 


Curriculum 

Component 

Component 

Total 

Cognitive  Level 

Total 

Process 

Skills 

Subject 

Matter 

Knowledge 

Application 

Matter  and  Energy 

4 ,5,  6, 
7,  8,  29, 
30,  31, 
32 

1,  2,  3, 
26,  27, 
28 

15  [30%] 

1,  26,  27 

2,  3,  4,  5, 
6,  7,  8,  28, 
29,  30,  31, 
32 

15  [30%] 

Living  Things  and 
Environment 

12,  13, 
14,  15, 
35,  37, 
38,  39, 
40 

9,  10, 
11,33, 
34,  36 

15  [30%] 

9,  10,  33 

11,  12,  13, 
14,  15,  34, 
35,  36,  37, 
38,  39,  40 

15  [30%] 

Earth,  Space,  and 
Time 

16,  42 

17,41 

4 [8%] 

17 

16,  41,  42 

4 [8%] 

Process  Skills 
Independent  of 
Prescribed 
Subject  Matter 

18,  19, 
20,  21, 
22,  23, 
24,  25, 
43,  44, 
45,  46, 
47,  48, 
49,  50 

16  [32%] 

18,  45,  46 

19,20,  21, 
22,  23,  24, 
25,  43,  44, 
47,  48,  49, 
50 

16  [32%] 

Total 

36  [72%] 

14  [28%] 

50  [100%] 

10  [20%] 

40  [80%] 

50  [100%] 
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STUDENT  PARTICIPATION 


In  June  1991,  principals  reported  a total  absent,  and  who  were  exempted.  Results  for 

population  of  39  216  students  in  Grade  3.  students  in  French  Immersion  or  Francoph- 

Table  2-2  presents  the  number  and  percent-  one  programs  are  reported  separately  to 
age  distribution  of  students  who  wrote  the  participating  schools  and  jursidictions. 

Grade  3 Science  Achievement  Test,  who  were 

Table  2-2 
Grade  3 Science 
Student  Participation 
June  1991 


Category 

Number 
of  Students 

Percentage 
of  Students 

TotalNumber  of  Students  In  Regular  Programs* 

Wt-S  33  83? 

86.3 

; Students  Who  Wrote  the  Regular  Form** 

30  427 

77.6 

Students Who  Wrote  the  Achievement-Over-Time  Forms: 

; 3 410 

8.7 

as  the  PfegtS  sir  SNdbrm 

1169 

3.0 

1122 

' 2:9  :: 

:£•••  • Form  F (Fuchsia)  " 

1119 

2,9 

Other  Students  Who  Wrote:*** 

988 

2.5 

Francophone  Students — Wrote  in  French 

0 

0.0 

Francophone  Students — Wrote  in  English 

51 

0.1 

French  Immersion  Students — Wrote  in  French 

0 

0.0 

French  Immersion  Students — Wrote  in  English 

471 

1.2 

Language-of-Testing  Students — Wrote  in  French 

232 

0.6 

Language-of-Testing  Students — Wrote  in  English 

234 

0.6 

Students  Absent 

1 502 

3.8 

Students  Exempted  from  Writing 

2 889 

7.4 

Categories  of  Exemption: 

1.  Special  Needs  Students 

1 014 

2.6 

2.  Subject  Was  Not  Taught  This  Term 

337 

0.9 

3.  ESL  Students 

548 

1.4 

4.  Language  of  Instruction  Was  Not  English 

805 

2.1 

5.  Other  (as  approved  by  the  Superintendent) 

185 

0.5 

Total  Principals’  Reported  Population: 

Test  Day,  June  1991 

39  216 

100.0 

Grade  3 Enrolment:  September  30,  1990 

41  001 

♦Provincial  results  are  based  on  test  scores  achieved  by  students  who  were  in  the  “regular”  (Mathematics  or  Science 
or  Social  Studies)  programs.  Such  students  are  defined  as  those  who  were  instructed  in  English  and  wrote  the 
English  form  of  the  achievement  test,  and  those  students  who  were  instructed  in  a language  other  than  English  or 
French  and  wrote  the  English  form  of  the  test.  These  students  are  indicated  by  the  shaded  area  of  the  table. 

♦♦Regular  Form  refers  to  the  unmodified  1991  test.  The  modified  tests  are  described  in  Section  8 of  this  report. 

♦♦♦Results  for  Francophone  or  French  Immersion  students  or  for  students  participating  in  the  language-of-testing 
study  are  reported  directly  to  participating  schools  and  jurisdictions  and  are  not  calculated  in  the  provincial 
results. 
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RESULTS  IN  RELATION  TO  STANDARDS 


Figures  2-1  and  2-2  and  Table  2-3  show  the  percentage  of  students  achieving  the  acceptable 
standards  and  the  standards  of  excellence  on  components  of  the  test.  The  levels  of  performance 
were  higher  than  expected  for  the  standards  of  excellence  and  lower  than  expected  for  the 
acceptable  standards. 

Figure  2-1 

GRADE  3 SCIENCE 

PERCENTAGE  OF  STUDENTS  ACHIEVING  ACCEPTABLE 
STANDARDS  OR  HIGHER  ON  COMPONENTS  OF  THE  TEST 


*the  percentage  of  students  in  the  province  expected  to  meet  the  acceptable  standards  or  higher 

**the  percentage  of  students  in  the  province  who  met  the  acceptable  standards  or  higher 


Figure  2-2 


GRADE  3 SCIENCE 

PERCENTAGE  OF  STUDENTS  ACHIEVING  STANDARDS  OF 
EXCELLENCE  ON  COMPONENTS  OF  THE  TEST 


Achievement  Standards* ** 
(See  Appendix  A) 


[ [Actual  Results** 


*the  percentage  of  students  in  the  province  expected  to  meet  the  standards  of  excellence 
**the  percentage  of  students  in  the  province  who  met  the  standards  of  excellence 
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Table  2-3 
Grade  3 Science 

Students  Achieving  Standards  on  Components  of  the  Test 

June  1991 


Reporting  Category 

Maximum 

Possible 

Score 

Provincial 
Assessment 
Standard* 
(Raw  Score) 

Provincial 
Achievement 
Standard** 
(Per  Cent) 

Expected 

Number 

Actual 

Number 

Actual 
Per  Cent 

Standard  of  Excellence 

Both  Major  Components*** 

36  + 14 

32  + 12 

10 

3 384 

3 731 

11.0 

Process  Skills  Only 

36 

32 

15 

5 076 

6 134 

18.1 

Subject  Matter  Only 

14 

12 

15 

5 076 

7 147 

21.1 

Acceptable  Standard 
or  Higher 

Both  Major  Components*** 

36  + 14 

19+8 

80 

27  070 

23  712 

70.1 

Process  Skills  Only 

36 

19 

85 

28  761 

27  240 

80.5 

Subject  Matter  Only 

14 

8 

85 

28  761 

25  546 

75.5 

Below  Acceptable  Standard 

N/A 

N/A 

N/A 

N/A 

4 763 

14.1 

on  Both  Components 

*The  Provincial  Assessment  Standard  is  a score  determined  by  appropriate  standard-setting  procedures  and  is  the 
lowest  score  students  must  achieve  for  their  performance  to  be  judged  “acceptable”  and/or  “excellent”  in  relation 
to  curricular  expectations.  See  Appendix  A. 


**The  Provincial  Achievement  Standard  refers  to  the  percentage  of  students  expected  to  meet  or  exceed  the 
Provincial  Assessment  Standard.  See  Appendix  A. 

***In  previous  years,  a standard  for  the  total  test  was  reported.  An  individual  could  meet  the  standard  for  the  test 
overall  but  might  not  achieve  the  standard  for  both  components  of  the  test.  The  new  method  of  reporting  allows 
for  a more  meaningful  interpretation  of  the  results.  To  allow  comparison  with  results  from  previous  years,  the 
total  test  score  is  reported  in  Table  2-4. 


-11- 


Table  2-4  shows  the  percentage  of  students  achieving  the  acceptable  standard  and  the  stand- 
ard of  excellence  on  the  total  test.  These  levels  of  performance  exceeded  the  expectations  for 
the  standard  of  excellence  but  were  lower  than  expected  for  the  acceptable  standard. 

Table  2-4 
Grade  3 Science 

Students  Achieving  Standards  on  the  Total  Test 
June  1991 


Reporting  Category 

Maximum 

Possible 

Score 

Provincial 
Assessment 
Standard* 
(Raw  Score) 

Provincial 
Achievement 
Standard** 
(Per  Cent) 

Expected 

Number 

Actual 

Number 

Actual 
Per  Cent 

Total  Test: 

Standard  of  Excellence 

50 

44 

15 

5 076 

5 606 

16.6 

Acceptable  Standard 
or  Higher 

50 

27 

85 

28  761 

26  710 

78.9 

*The  Provincial  Assessment  Standard  is  a score  determined  by  appropriate  standard-setting  procedures  and  is  the 
lowest  score  students  must  achieve  for  their  performance  to  be  judged  “acceptable”  and/or  “excellent”  in  relation  to 
curricular  expectations.  See  Appendix  A. 

**The  Provincial  Achievement  Standard  refers  to  the  percentage  of  students  expected  to  meet  or  exceed  the  Provin- 
cial Assessment  Standard.  See  Appendix  A. 


It  should  be  noted  that  the  actual  percentages 
for  Grade  3 Science  (16.6  and  78.9)  are  based 
on  the  33  837  students  in  the  regular  pro- 
grams who  wrote  the  test. 

If,  however,  the  percentages  are  based  on  the 
total  population  reported  by  principals 
(39  216),  less  the  French  Immersion  and 
Francophone  students  who  wrote  the  test 
(988),  the  percentages  would  be: 

14.7%  achieving  the  standard  of  excellence 
69,9%  achieving  the  acceptable  standard 

If  the  percentages  are  based  on  the  Septem- 
ber 30,  1990,  Grade  3 enrolment  (41  001),  less 
the  French  Immersion  and  Francophone 
students  who  wrote  the  test  (988),  the  per- 
centages would  be: 

14.0%  achieving  the  standard  of  excellence 
66.8%  achieving  the  acceptable  standard 

It  is  emphasized  that  the  above  percentages, 
based  on  the  total  population  and  enrolment 
figures,  present  the  worst  possible  scenario. 


It  is  highly  unlikely  that  none  of  the  students 
who  was  absent,  exempt,  or  not  accounted  for 
would  have  achieved  standards.  The  absence 
of  information  on  these  students  is  nonethe- 
less problematic. 

The  number  of  students  achieving  the  accept- 
able standard  and  the  standard  of  excellence 
for  each  jurisdiction  was  analysed  to  deter- 
mine whether  jurisdictions  were  below,  meet- 
ing, or  exceeding  provincial  achievement 
standards.  Jurisdictions  classified  as  meeting 
provincial  achievement  standards  were  those 
for  which  the  difference  between  the  actual 
number  of  students  and  the  expected  number 
of  students  at  or  above  standards  was  not 
statistically  significant.  A 95%  confidence 
interval  was  used;  this  criterion  means  that 
differences  are  only  reported  when  there  is  a 
5%  or  smaller  probability  that  a difference  of 
that  size  could  occur  by  chance.  The  results 
are  reported  in  Tables  2-5  and  2-6.  The 
percentages  in  the  tables  are  based  on  200 
jurisdictions  (including  private  schools ). 
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Table  2-5 
Grade  3 Science 

Percentage  Distribution  of  Jurisdictions*  Meeting 
Provincial  Achievement  Standards  on  Components  of  the  Test 

June  1991 


Component 

Jurisdictions  Not 
Meeting  Provincial 
Achievement  Stan- 
dard (Per  Cent) 

Jurisdictions 
Meeting  Provincial 
Achievement  Stan- 
dard (Per  Cent) 

Jurisdictions  Ex- 
ceeding Provincial 
Achievement  Stan- 
dard (Per  Cent) 

Standard  of  Excellence 

Both  Major  Components 

7.0 

71.0 

22.0 

Process  Skills  Only 

6.5 

66.5 

27.0 

Subject  Matter  Only 

3.5 

58.0 

38.5 

Acceptable  Standard  or  Higher 

Both  Major  Components 

48.0 

48.0 

4.0 

Process  Skills  Only 

33.0 

60.5 

6.5 

Subject  Matter  Only 

47.5 

51.0 

1.5 

Table  2-6 
Grade  3 Science 

Percentage  Distribution  of  Jurisdictions*  Meeting 
Provincial  Achievement  Standards  on  the  Total  Test 
June  1991 


Reporting  Category 

Jurisdictions 
Not  Meeting 
Provincial 
Achievement 
Standard 
(Per  Cent) 

Jurisdictions 
Meeting  Prov- 
incial Achieve- 
ment Standard 
(Per  Cent) 

Jurisdictions 

Exceeding 

Provincial 

Achievement 

Standard 

(Per  Cent) 

Total  Test: 

Standard  of  Excellence 

9.0 

66.0 

25.0 

Acceptable  Standard  or  Higher 

37.5 

56.5 

6.0 

* Jurisdictions  with  fewer  than  six  students  are  excluded  because  the  statistical  significance  of  the  difference 
between  the  number  actually  meeting  the  standard  and  the  number  expected  to  meet  the  standard  when 
calculated  and  reported  is  not  educationally  meaningful. 
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REPORTING  CATEGORIES 


Table  2-7  shows  the  total  marks  possible  and 
the  provincial  raw  score  results  for  the 
reporting  categories  of  the  Grade  3 Science 
Achievement  Test. 

It  is  important  to  stress  that  the  averages  in 
the  various  reporting  categories  cannot  be 
directly  compared  with  one  another.  Rather, 
the  results  shown  in  Table  2-7  can  best  be 
used  in  conjunction  with  parallel  tables  in  the 


jurisdiction  and  school  reports.  Variations  in 
patterns  of  students’  responses  can  help  indi- 
cate strengths  and  weaknesses  in  local  educa- 
tional programs. 

Statistics  presented  in  this  table  are  based  on 
results  achieved  by  31  596  students  (those  who 
wrote  the  regular  form  and  those  who  wrote 
Form  G). 


Table  2-7 
Grade  3 Science 

Raw  Score  Results  by  Reporting  Category 
June  1991 


Reporting  Category 

Maximum 
Possible 
Raw  Score 

Raw  Score 
Average 

Raw  Score 
Standard  Deviation 

Total  Test 

50 

34.2 

9.1 

Major  Component: 

Process  Skills 

36 

24.8 

6.9 

Subject  Matter 

14 

9.4 

2.7 

Curricular  Elements: 

Matter  and  Energy 

15 

10.4 

2.7 

Living  Things  and  Environment 

15 

11.2 

2.8 

Earth,  Space,  and  Time 

4 

2.4 

1.1 

Process  Skills  as  Content 

16 

10.2 

3.9 

Cognitive  Level: 

Knowledge 

10 

6.7 

2.1 

Understanding  and  Application 

40 

27.5 

7.5 
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PERCENTAGE  OF  STUDENTS  CHOOSING  EACH  ALTERNATIVE 


Table  2-8  shows  the  percentage  of  students 
who  chose  each  alternative  (A,  B,  C,  and  D) 
for  each  multiple-choice  question.  The  correct 
response  (key)  for  each  question  is  also 
identified.  The  results  shown  in  Table  2-8  can 
best  be  used  in  conjunction  with  the  parallel 
tables  in  the  jurisdiction  and  school  reports. 


Variations  in  patterns  of  students’  responses 
to  questions  can  help  to  indicate  strengths  and 
weaknesses  in  local  educational  programs. 

Statistics  presented  in  this  table  are  based  on 
results  achieved  by  31  596  students  (those  who 
wrote  the  regular  form  and  those  who  wrote 
Form  G). 


Table  2-8 
Grade  3 Science 

Provincial  Results  for  Individual  Multiple-Choice  Questions* 

June  1991 


Item  Key 

Distribution  of 
Responses  (%) 

Item 

Key 

Distribution  of 
Responses  (%) 

A 

B 

C 

D 

A 

B 

C 

D 

1 

ill!! 

1.9 

III! 

89.0 

ilii 

26 

■ill 

14.0 

44.3 

31.1 

liiil 

2 

■11! 

7.3 

liiil 

11.2 

63*9 

27  1 

c 

5.9 

liiil 

70.2 

16*7 

3 

11®!! 

15.6 

ill 

11.5 

20*8 

28  I 

ilii 

6.8 

3,1 

86.6 

2,2 

4 1 

■sii 

62.7 

■ill 

3.1 

:;25*5: 

29 

B 

5.3 

iBil 

12.5 

3.9 

5 

0 

12.2 

v20M 

51.0 

ilii 

30 

D 

2.8 

10.6 

5.8 

6 1 

B 

9.9 

ilii 

22.6 

Bill 

31 

liiil 

6.6 

86,6 

2.5 

■si 

7 j 

B 

5.1 

80*8 

5.4 

■ii 

32  : 

■ill 

10.0 

8*9 

81.8 

2*6 

8 ; 

■111 

11.0 

70*0 

3.9 

14,2 

33 

■III 

8.0 

■ii! 

71.9 

■881 

9 i 

C 

3.6 

ilii! 

85.4 

■ii 

34 

1*1 

77.1 

6.0 

8.1 

■ii 

io  j 

■111 

3.7 

Ill 

4.6 

88*5: 

35  i 

D 

5.6 

■ii 

4.0 

ill! 

ii 

■in 

53.9 

12.0:: 

16.3 

16*7 : 

36 

■ill 

28.8 

. 2,3 

2.4 

ilii 

12 

■ill 

16.3 

■Ii 

1.7 

ilii 

37 

i*8 

13.3 

10.5 

28.5 

46,4 

13 

ill 

6.7 

■ii! 

8.2 

.176.1: 

38 

ilii 

71.5 

5.5 

4.1 

ill® 

14  ! 

A 

77.8 

: 9*4 

8.8 

2.3 

39 

■in 

77.5 

: 5*6 

5.5 

ilii! 

15  ; 

■III! 

4.8 

6*6 

84.2 

3.3 

40 

nil 

6.8 

80*6 

5.3 

6*2 

16  I 

■ill 

8.1 

ill® 

56.3 

19.2 

41 

1 Si 

19.3 

ilii! 

15.6 

ilii 

17  I 

mi 

12.0 

Ilii 

16.7 

10.1 

42 

11*1 

84.0 

3*7 

3.9 

■ii 

18  ! 

It® 

14.5 

■iii 

17.2 

ilii 

43 

B 

15.2 

Ilii! 

7.9 

■lit 

19  : 

A 

54.1 

ilii 

11.9 

■ilii 

44 

0 

6.6 

■ii 

9.6 

iii! 

20  1 

c 

3.4 

0.0 

76.7 

ilii 

45 

Hill 

16.6 

9,5 

63.0 

■ill 

21  j 

B 

9.7 

57,4 

6.1 

ilii 

46 

II® 

7.7 

ii#® 

14.9 

57,6 

22 

Hill! 

75.5 

4*5 

6.1 

■ill 

47 

■ill 

11.3 

15*1 

61.7 

III# 

23 

B':l 

3.4 

ilii 

3.8 

9<7 

48 

• AB 

61.3 

U% 

6.4 

.17*0 

24  j 

A 

63.1 

••5*6 

20.5 

5.3 

49 

i> 

12.9 

17*1 

11.8 

lUl! 

25 

■ii 

8.5 

68  2 : 

9.7 

6.4 

50 

A 

49.2 

15.0 

19.1 

10*8 

*The  sum  of  the  percentages  for  each  question  may  be  less  than  100%  because  the  No  Response  category 
is  not  included.  This  category  is  less  than  8%  for  any  of  these  questions. 
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GRADE  3 SCIENCE  EXAMINER’S  OBSERVATIONS 


The  following  is  a discussion  of  specific  areas  of  strength  and  of  difficulty  for  Grade  3 students. 
Sample  questions  are  provided  to  highlight  the  strengths  and  weaknesses  of  students  achiev- 
ing at  the  acceptable  standard  and  at  the  standard  of  excellence.  For  each  sample  question, 
the  circle  beside  or  under  the  correct  answer  has  been  filled  in.  The  percentage  of  students 
choosing  each  alternative  is  given  beside  each  circle. 

Acceptable  Standard — Sample  Questions  and  Commentary 


19.  Look  at  this  picture. 


28.  To  conserve  energy,  Jean  could 


6.8  O turn  up  the  heat  in  his  house 

3.1  O leave  the  refrigerator  door  open 

86.6  • turn  off  all  the  lights  when  he 
goes  to  bed 

2.2  O leave  the  television  on  when  he 
goes  out  to  play 

19.7  O just  been  sharpened 
11.9  O been  used  for  writing 
10.4  O a pink  eraser  on  the  end 


W> 


An  OBSERVATION  that  can  be  made  about  this 
pencil  is  that  it  has 

54.1  • a pointed  end 


40.  Which  group  of  plants  and  animals  are  found  in  an 
Alberta  forest  community? 


What  did  students  know  and  what  could  they  do? 

Generally,  these  students  had  an  understanding  of  basic 
process  skills.  They  could  classify  common  animals 
based  on  similar  physical  characteristics;  they  could 
identify  similarities  presented  in  a graph  or  a chart.  By 
observing,  they  could  predict  the  next  step  in  a sequence 
of  events  and  could  make  simple  interpretations. 

Specifically,  these  students  knew  that  when  materials 
are  heated,  they  occupy  more  space  and  can  increase  in 
size  (question  7).  As  well,  they  knew  that  common 
objects  made  of  specific  materials  may  or  may  not  have 
the  capacity  to  float  on  water  or  be  attracted  by  a mag- 
net. They  also  could  identify  some  common  plant  and 
animal  life  of  Alberta  (question  40)  and  knew  the  impor- 
tance of  conserving  energy  in  a home  (question  28). 
Students  knew  that  moss  and  mushrooms  are  types  of 
plants  that  grow  on  decaying  logs,  and  that  birds,  fish, 
and  reptiles  hatch  from  eggs.  They  understood  the 
energy  (food)  chains  that  exist  between  plants  and  ani- 
mals as  well  as  the  life  cycle  of  a plant. 
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8.  This  is  the  terrperature  today. 


What  did  students  find  difficult? 


arc 

Tore 

rrc 

arc 

-rrc 

-xrc 

-ace 


What  would  be  BEST  to  wear  outside  today? 


If 

Shorts  and  a t-shirt 

Winter  coat  and  mittens 

O 11.0 

• 70.0 

5 

Raincoat  and  rubber  boots 

Jogging  suit  and  running 

shoes 

O 3.9  O 14.2 


These  students  had  difficulty  applying  their  science 
knowledge  to  new  situations  or  to  contexts  that  went 
beyond  familiar  plants  and  animals.  Generally,  they 
could  not  sort  out  extraneous  information  or  apply 
their  observation  skills  objectively. 

In  question  8,  for  example,  students  achieving  at  the 
acceptable  standard  had  difficulty  making  the  neces- 
sary connections  between  temperature  measure- 
ments and  the  conditions  that  exist  in  the  environ- 
ment. In  question  19  (see  the  previous  page),  stu- 
dents were  asked  to  make  a direct  observation  about 
the  pencil  in  the  picture.  Their  responses  indicated 
that  they  tended  to  rely  on  what  they  knew  a pencil 
looked  like  rather  than  on  what  they  could  see  in  the 
picture. 


Standard  of  Excellence — Sample  Questions  and  Commentary 


13.  Tom  predicted  that  radish  seeds  grow  better  in  sand 
than  in  garden  soil. 

In  each  pot,  he  planted  six  radish  seeds. 


Sand  Garden  soil 

Which  picture  shows  how  Tom  SHOULD  test  his  prediction? 


0 8.2  •76.1 


What  did  students  know  and  what  could  they  do? 

Students  achieving  at  the  standard  of  excellence 
knew  that  a specific  unit  of  measurement  is  neces- 
sary to  make  accurate  measurements  and  that  the 
senses  of  touch  and  sight  can  be  used  to  estimate  the 
length  of  an  object.  They  also  knew  which  variables 
to  control  when  experimenting  with  plant  growth 
(question  13).  These  students  also  knew  that  metals 
expand  when  heated  and  that  the  phase  changes  of 
water  can  be  reversed  whereas  the  process  of  burn- 
ing paper  cannot.  In  addition,  they  understood  the 
interrelationship  that  exists  between  living  things 
and  their  environment  (question  25)  and  how  sym- 
bols can  be  used  to  represent  changes  of  events  that 
occur  in  nature. 
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25.  Some  trees  in  an  area  were  cut  down  to  make  room  for  a 
baseball  diamond. 


What  is  the  MOST  likely  result  of  this  change? 

8.5  O There  are  more  shady  places. 

68.2  • There  are  fewer  birds  in  the  area. 


These  students  had  good  observation  and  inference 
skills.  They  could  read  an  outdoor  thermometer,  and 
they  could  order  objects  from  lightest  to  heaviest  based 
on  their  mass.  These  students  could  also  classify 
unfamiliar  living  and  nonliving  things  based  on  their 
observations  of  physical  characteristics,  apply  general 
knowledge  to  solve  novel  problems,  transfer  new 
learnings  gained  from  interpreting  graphs  (question 
24),  and  analyse  new  situations. 

What  did  students  find  difficult? 


9.7  O Children  have  less  room  to  play. 

6.4  O More  squirrels  have  built  homes  in  the  area. 


24.  Five  students  recorded  the  number  of  floor  hockey 
games  each  of  them  played  in  one  year. 

FLOOR  HOCKEY  GAMES  PLAYED 


Players 

Which  student  played  40  more  games  than  Jackie? 

63.1  • Ted 
5.6  O Dana 
20.5  O Tim 
5.3  O Stan 


Analysis  indicates  that  students  achieving  at  the 
standard  of  excellence  did  not  have  any  apparent 
difficulties  with  Grade  3 Science  knowledge  or  process 
skills. 


Teachers'  Review  of  the  Test 

The  Grade  3 Science  teachers  who  reviewed  the  test 
felt  that  it  was  fair  but  tough.  As  well,  they  felt  that 
the  concepts  and  skills  tested  reflected  the  learning 
expectations  of  the  Division  I Science  Program  very 
well.  Teachers  also  liked  the  use  of  questions  that 
required  students  to  think  and  apply  their  knowledge 
of  science  to  the  world  they  experience  beyond  the 
classroom. 
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SECTION  3 

GRADE  6 MATHEMATICS 


GENERAL  DESCRIPTION 


The  Grade  6 Mathematics  Achievement  Test 
was  divided  into  three  booklets.  Booklet  1 
contained  36  questions,  and  students  were 
allowed  to  use  a calculator;  Booklet  2 
contained  19  questions  and  students  were  not 
allowed  to  use  a calculator;  Booklet  3 
contained  five  2-minute  tests  of  48  number 
fact  questions  each,  and  students  were  not 
allowed  to  use  a calculator.  The  time  allotted 
was  55  minutes  for  Booklet  1 and  30  mintues 
for  Booklet  2. 

Statistics  for  the  total  test  and  for  components 
in  Booklets  1 and  2 are  based  on  the  results 
achieved  by  31  930  students  (28  649  wrote  the 
regular  form,  and  3 281  participated  in  the 
achievement-over-time  study,  as  shown  in 
Table  3-3). 


This  section  of  the  report  provides  answers  to 

the  following  questions: 

• How  many  Grade  6 students  wrote  each 
form  of  the  test  or  were  absent  and  exempt? 

• What  percentage  of  Grade  6 students 
attained  the  acceptable  standard  in 
Mathematics  according  to  provincial 
criteria? 

• What  percentage  of  Grade  6 students 
attained  the  standard  of  excellence  in 
Mathematics  according  to  provincial 
criteria? 

• What  did  Grade  6 students  know  and  what 
could  they  do  in  Mathematics? 

• What  parts  of  the  Mathematics  curriculum 
caused  Grade  6 students  difficulty? 


SUMMARY  OF  RESULTS  FOR  BOOKLETS  1 AND  2 
Results  in  Relation  to  Standards 

Results  show  that  68.9%  of  students  who  wrote  the  test  achieved  the  acceptable  standards  on 
both  components  and  14.3%  achieved  the  standards  of  excellence  on  both  components.  These 
results  were  lower  than  expected  for  the  acceptable  standards  but  higher  than  expected  for  the 
standards  of  excellence. 

Average  Score 

The  average  total  test  score  for  booklets  1 and  2 was  68.2%,  with  a standard  deviation  of  19.1. 
The  average  raw  score  was  37.5  marks  out  of  a possible  55,  with  a standard  deviation  of  10.5. 


CONTENT  OF  THE  TEST 


The  Grade  6 Mathematics  Achievement  Test 
was  designed  to  reflect  the  Grade  6 
Mathematics  Curriculum  Specifications 
(revised  May  1986)  that  were  developed  from 
the  revised  1982  Mathematics  Program  of 
Studies.  The  scope  of  the  test  was  limited  to 
curriculum  objectives  that  could  be  efficiently 
measured  on  a paper  and  pencil  test. 
Questions  on  the  test  were  drawn  from  the 
content  of  two  major  components  in  the  core 
program: 


• Subject  Matter 

• Problem  Solving 

The  subject  matter  component  consisted  of 
questions  associated  with  the  six  content 
areas:  numeration,  operations  and  proper- 
ties, measurement,  geometry,  graphing,  and 
problem-solving  strategies.  The  problem- 
solving component  consisted  of  questions 
integrated  with  content  areas  and  questions 
independent  of  subject  matter. 
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Questions  on  the  Grade  6 Mathematics 
Achievement  Test  measured  student  achieve- 
ment at  three  cognitive  levels: 

• Knowledge — recognize  or  recall  facts,  defini- 
tions, rules,  procedures,  and  perform  rou- 
tine manipulation 

• Comprehension — demonstrate  understand- 
ing of  principles  and  concepts;  translate 


from  concrete  to  pictorial  to  symbolic  repre- 
sentations or  translate  in  the  reverse  order 

• Application  and  Problem  Solving — solve 
problems  utilizing  learned  skills  and  con- 
cepts; recognize  patterns  and  relationships; 
compare,  interpret,  and  analyse  data  by 
applying  learned  skills  and  knowledge. 


ACHIEVEMENT  TEST  BLUEPRINT 

Table  3-1  presents  the  blueprint  used  to  develop  booklets  1 and  2 of  the  Grade  6 Mathematics 
Achievement  Test.  Classification  of  each  question  by  component,  cognitive  level,  and  subtest  is 
indicated  in  the  table. 

Table  3-1 

Grade  6 Mathematics 

Achievement  Test  Blueprint,  Booklets  1 and  2 
June  1991 


Curricular 

Elements 

Component 

Total 

Cognitive  Level 

Total 

Problem 

Solving 

Subject 

Matter 

Knowledge 

Compre- 

hension 

Application 
and  Problem 
Solving 

Numeration 

4,  53,  54, 
55 

1,  2,  3, 

5,  6,  14, 
15,  38,  39 

13  [23.6%] 

1,  2,  14, 
38,  39 

3,  5, 
6,  15 

4,  53,  54, 
55 

13  [23.6%] 

Operations 
and  Properties 

20,  27,  36, 
41,  47,  48, 
50 

29,  37,  40, 
42,  43,  44, 
45,  46,  49 

16  [29.1%] 

42 

29,  36, 
37  ,40, 
43,  44, 
45,  46, 
49 

20,  27,  41, 
47,  48,  50 

16  [29.1%] 

Measurement 

16,  28,  35, 
51,52 

7,  8,  11, 
12,  13,  17 

11  [20.0%] 

7,  12,  13, 
17 

8,  11 

16,  28,  35, 
51,52 

11  [20.0%] 

Geometry 

32 

9,  30,  31, 
33,  34 

6 [10.9%] 

9,  30 

31,34 

32,  33 

6 [10.9%] 

Graphing 

21,  22,  23, 
24,  25 

26 

6 [10.9%] 

— 

24,  26 

21,  22,  23, 
25 

6 [10.9%] 

Problem-Solving 

Strategies 

10,  18,  19 

— 

3 [5.5%] 

— 

— 

10,  18,  19 

3 [5.5%] 

Total 

25  [45.5%] 

30  [55.5%] 

55  [100%] 

12  [21.8%] 

19  [34.5%] 

24  [43.6%] 

55  [100%] 
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Table  3-2  presents  the  blueprint  used  to  develop  Booklet  3 of  the  Grade  6 Mathematics 
Achievement  Test.  This  booklet  consisted  of  five  timed  tests,  measuring  student  performance 
in  operations  involving  sums  and  minuends  to  18,  and  products  and  dividends  to  81. 


Table  3-2 

Grade  6 Mathematics 
Achievement  Test  Blueprint,  Booklet  3 
Number  Facts:  Speed  and  Accuracy  Tests 
June  1991 


Operations 

Time 
Allotted 
in  Minutes 

Number 

of 

Questions 

Addition 

2 

48 

Subtraction 

2 

48 

Multiplication 

2 

48 

Division 

2 

48 

Mixed  Operations 

2 

48 
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STUDENT  PARTICIPATION 


In  June  1991,  principals  reported  a total 
population  of  36  565  students  in  Grade  6. 
Table  3-3  presents  the  number  and  percent- 
age distribution  of  students  who  wrote  the 
Grade  6 Mathematics  Achievement  Test, 


who  were  absent,  and  who  were  exempted. 
Results  for  students  in  French  Immersion  or 
Francophone  programs  are  reported  sepa- 
rately to  participating  schools  and  jurisdic- 
tions. 


Table  3-3 

Grade  6 Mathematics 
Student  Participation 

June  1991 


Category 

Number 
of  Students 

Percentage 
of  Students 

Total  ! i>f  Students  in  Regular  Programs* 

31930 

37.3 

. 

28  649 

73.4 

Students  Who  Wrote  the  Achievement-Over-Time  Forms: 

3 231 

0.0 

F as  the  Regular  Form 

SB 

3.1 

F<  ■ . men)  ’ ••  • v . ••••:  r:'!v 

1079 

3.0 

Form  L (Lime) 

l 3.0 

Other  Students  Who  Wrote:** *** 

1 884 

5.1 

Francophone  Students — Wrote  in  French 

206 

0.6 

Francophone  Students — Wrote  in  English 

0 

0.0 

French  Immersion  Students — Wrote  in  French 

1057 

2.9 

French  Immersion  Students — Wrote  in  English 

45 

0.1 

Language-of-Testing  Students — Wrote  in  French 

283 

0.8 

Language-of-Testing  Students — Wrote  in  English 

293 

0.8 

Students  Absent 

1304 

3.6 

Students  Exempted  from  Writing 

1447 

4.0 

Categories  of  Exemption: 

1.  Special  Needs  Students 

934 

2.6 

2.  Subject  Was  Not  Taught  This  Term 

14 

0.0 

3.  ESL  Students 

235 

0.6 

4.  Language  of  Instruction  Was  Not  English 

120 

0.3 

5.  Other  (as  approved  by  the  Superintendent) 

144 

0.4 

Total  Principals’  Reported  Population: 

Test  Day,  June  1991 

36  565 

100.0 

Grade  6 Enrolment:  September  30,  1990 

37  114 

* Provincial  results  are  based  on  test  scores  achieved  by  students  who  were  in  the  “regular”  (Mathematics  or  Science  or 
Social  Studies)  programs.  Such  students  are  defined  as  those  who  were  instructed  in  English  and  wrote  the  English 
form  of  the  achievement  test,  and  those  students  who  were  instructed  in  a language  other  than  English  or  French  and 
wrote  the  English  form  of  the  test.  These  students  are  indicated  by  the  shaded  area  of  the  table. 

**  Regular  Form  refers  to  the  unmodified  1991  test.  The  modified  tests  are  described  in  Section  8 of  this  report. 

***Results  for  Francophone  or  French  Immersion  students  or  for  students  participating  in  the  language-of-testing  study 
are  reported  directly  to  participating  schools  and  jurisdictions  and  are  not  calculated  in  the  provincial  results. 
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RESULTS  FOR  BOOKLETS  1 AND  2 


Figures  3-1  and  3-2  and  Table  3-4  show  the  percentage  of  students  achieving  the  acceptable 
standards  and  the  standards  of  excellence  on  components  of  the  test.  The  levels  of  performance 
were  higher  than  expected  for  the  standards  of  excellence  but  lower  than  expected  for  the 
acceptable  standards. 


Figure  3-1 

GRADE  6 MATHEMATICS 

PERCENTAGE  OF  STUDENTS  ACHIEVING  ACCEPTABLE 
STANDARDS  OR  HIGHER  ON  COMPONENTS  OF  THE  TEST 
June  1991 

100-i 


Solving  Matter  Components 


^ Achievement  Standards* 
(See  Appendix  A) 


111!  Actual  Results** 


*the  percentage  of  students  in  the  province  expected  to  meet  the  acceptable  standards  or  higher 

**the  percentage  of  students  in  the  province  who  met  the  acceptable  standards  or  higher 


Figure  3-2 

GRADE  6 MATHEMATICS 

PERCENTAGE  OF  STUDENTS  ACHIEVING  STANDARDS 
OF  EXCELLENCE  ON  COMPONENTS  OF  THE  TEST 
June  1991 


*the  percentage  of  students  in  the  province  expected  to  meet  the  standards  of  excellence 
**the  percentage  of  students  in  the  province  who  met  the  standards  of  excellence 
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Table  3-4 

Grade  6 Mathematics:  Booklets  1 and  2 
Students  Achieving  Standards  on  Components  of  the  Test 

June  1991 


Reporting  Category 

Maximum 

Possible 

Score 

Provincial 
Assessment 
Standard* 
(Raw  Score) 

Provincial 
Achievement 
Standard** 
(Per  Cent) 

Expected 

Number 

Actual 

Number 

Actual 
Per  Cent 

Standard  of  Excellence 

Both  Major  Components*** 

25  + 30 

21  + 27 

10 

3 193 

4 572 

14.3 

Problem  Solving  Only 

25 

21 

15 

4 789 

7 508 

23.5 

Subject  Matter  Only 

30 

27 

15 

4 789 

6 426 

20.1 

Acceptable  Standard 
or  Higher 

Both  Major  Components*** 

25  + 30 

13  + 17 

80 

25  544 

22  004 

68.9 

Problem  Solving  Only 

25 

13 

85 

27  140 

24  088 

75.4 

Subject  Matter  Only 

30 

17 

85 

27  140 

24  457 

76.6 

Below  Acceptable  Standard 

N/A 

N/A 

N/A 

N/A 

5 389 

16.9 

on  Both  Components 

* The  Provincial  Assessment  Standard  is  a score  determined  by  appropriate  standard-setting  procedures  and  is  the 
lowest  score  students  must  achieve  for  their  performance  to  be  judged  “acceptable”  and/or  “excellent”  in  relation  to 
curricular  expectations.  See  Appendix  A. 


**The  Provincial  Achievement  Standard  refers  to  the  percentage  of  students  expected  to  meet  or  exceed  the  Provincial 
Assessment  Standard.  See  Appendix  A. 

***  In  previous  years,  a standard  for  the  total  test  was  reported.  An  individual  could  meet  the  standard  for  the  test 
overall  but  might  not  achieve  the  standard  for  both  components  of  the  test.  The  new  method  of  reporting  allows  for  a 
more  meaningful  interpretation  of  the  results.  To  allow  comparison  with  results  from  previous  years,  the  total  test 
score  is  reported  in  Table  3-5. 
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Table  3-5  shows  the  percentage  of  students  achieving  the  acceptable  standard  and  the  stand- 
ard of  excellence  on  the  total  test.  These  levels  of  performance  were  higher  than  expected  for 
the  standard  of  excellence  but  were  lower  than  expected  for  the  acceptable  standard. 

Table  3-5 

Grade  6 Mathematics:  Booklets  1 and  2 
Students  Achieving  Standards  on  the  Total  Test 

June  1991 


Reporting  Category 

Maximum 

Possible 

Score 

Provincial 
Assessment 
Standard* 
(Raw  Score) 

Provincial 
Achievement 
Standard** 
(Per  Cent) 

Expected 

Number 

Actual 

Number 

Actual 
Per  Cent 

Total  Test: 

Standard  of  Excellence 

55 

48 

15 

4 789 

6 566 

20.6 

Acceptable  Standard 
or  Higher 

55 

30 

85 

27  140 

24  118 

75.5 

*The  Provincial  Assessment  Standard  is  a score  determined  by  appropriate  standard-setting  procedures  and  is  the 
lowest  score  students  must  achieve  for  their  performance  to  be  judged  “acceptable”  and/or  “excellent”  in  relation 
to  curricular  expectations.  See  Appendix  A. 

**The  Provincial  Achievement  Standard  refers  to  the  percentage  of  students  expected  to  meet  or  exceed  the  Provin- 
cial Assessment  Standard.  See  Appendix  A. 


It  should  be  noted  that  the  actual  percentages 
for  Grade  3 Mathematics  (20.6  and  75.5)  are 
based  on  the  31  930  students  in  the  regular 
programs  who  wrote  the  test. 

If,  however,  the  percentages  are  based  on  the 
total  population  reported  by  principals 
(36  565),  less  the  French  Immersion  and 
Francophone  students  who  wrote  the  test 
(1  884),  the  percentages  would  be: 

18.9%  achieving  the  standard  of  excellence 
69.5%  achieving  the  acceptable  standard 

If  the  percentages  are  based  on  the  Septem- 
ber 30,  1990,  Grade  6 enrolment  (37  114),  less 
the  French  Immersion  and  Francophone 
students  who  wrote  the  test  (1  884),  the 
percentages  would  be: 

18.6%  achieving  the  standard  of  excellence 
68.5%  achieving  the  acceptable  standard 

It  is  emphasized  that  the  above  percentages, 
based  on  the  total  population  and  enrolment, 


present  the  worst  possible  scenario.  It  is  highly 
unlikely  that  none  of  the  students  who  was  absent, 
exempt,  or  not  accounted  for  would  have  achieved 
standards.  The  absence  of  information  on  these 
students  is  nonetheless  problematic. 

The  number  of  students  achieving  the  acceptable 
standard  and  the  standard  of  excellence  for  each 
jurisdiction  were  analysed  to  determine  whether 
jurisdictions  were  below,  meeting,  or  exceeding 
provincial  achievement  standards.  Jurisdictions 
classified  as  meeting  provincial  achievement 
standards  were  those  for  which  the  difference 
between  the  actual  number  of  students  and  the 
expected  number  of  students  at  or  above  standards 
was  not  statistically  significant.  A 95%  confidence 
interval  was  used;  this  criterion  means  that  differ- 
ences are  only  reported  when  there  is  a 5%  or 
smaller  probability  that  a difference  of  that  size 
could  occur  by  chance.  The  results  are  reported  in 
tables  3-6  and  3-7.  The  percentages  in  the 
tables  are  based  on  202 jurisdictions  ( includ- 
ing private  schools ). 
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Table  3-6 

Grade  6 Mathematics:  Booklets  1 and  2 
Percentage  Distribution  of  Jurisdictions*  Meeting 
Provincial  Achievement  Standards  on  Components  of  the  Test 

June  1991 


Component 

Jurisdictions 
Not  Meeting 
Provincial 
Achievement 
Standard 
(Per  Cent) 

Jurisdictions 
Meeting  Prov- 
incial Achieve- 
ment Standard 
(Per  Cent) 

Jurisdictions 

Exceeding 

Provincial 

Achievement 

Standard 

(Per  Cent) 

Standard  of  Excellence 

Both  Major  Components 

3.0 

73.2 

23.8 

Problem  Solving  Only 

2.0 

59.9 

38.1 

Subject  Matter  Only 

5.4 

69.8 

24.8 

Acceptable  Standard  or  Higher 

Both  Major  Components 

57.9 

40.1 

2.0 

Problem  Solving  Only 

54.4 

44.6 

1.0 

Subject  Matter  Only 

51.0 

47.5 

1.5 

Table  3-7 

Grade  6 Mathematics:  Booklets  1 and  2 
Percentage  Distribution  of  Jurisdictions*  Meeting 
Provincial  Achievement  Standards  on  the  Total  Test 

June  1991 


Reporting  Category 

Jurisdictions 
Not  Meeting 
Provincial 
Achievement 
Standard 
(Per  Cent) 

Jurisdictions 
Meeting  Prov- 
incial Achieve- 
ment Standard 
(Per  Cent) 

Jurisdictions 

Exceeding 

Provincial 

Achievement 

Standard 

(Per  Cent) 

Total  Test: 

Standard  of  Excellence 

4.0 

69.8 

26.2 

Acceptable  Standard  or  Higher 

56.4 

42.6 

1.0 

*Jurisdictions  with  fewer  than  six  students  are  excluded  because  the  statistical  significance  of  the  difference 
between  the  number  actually  meeting  the  standard  and  the  number  expected  to  meet  the  standard  when 
calculated  and  reported  is  not  educationally  meaningful. 
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Reporting  Categories 


Table  3-8  shows  the  total  marks  possible  and 
the  provincial  raw  score  results  for  the  report- 
ing categories  of  Booklets  1 and  2 of  the 
Grade  6 Mathematics  Achievement  Test. 

It  is  important  to  stress  that  the  averages  on 
the  various  reporting  categories  cannot  be 
directly  compared  with  one  another.  Rather, 
the  results  shown  in  Table  3-8  can  best  be 
used  in  conjunction  with  parallel  tables  in  the 


jurisdiction  and  school  reports.  Variations  in 
patterns  of  students’  responses  to  questions 
can  help  to  indicate  strengths  and  weak- 
nesses in  local  educational  programs. 

Statistics  presented  in  this  table  are  based  on 
results  achieved  by  29  768  students  (those 
who  wrote  the  regular  form  and  those  who 
wrote  Form  B). 


Table  3-8 

Grade  6 Mathematics:  Booklets  1 and  2 
Raw  Score  Results  by  Reporting  Category 
June  1991 


Reporting  Category 

Maximum 

Possible 

Score 

Raw  Score 
Average 

Raw  Score 
Standard  Deviation 

Total  Test 

55 

37.5 

10.6 

Msgor  Component: 

Problem  Solving 

25 

16.4 

5.3 

Subject  Matter 

30 

21.0 

5.8 

Curricular  Elements: 

Numeration 

13 

8.1 

3.2 

Operations  and  Properties 

16 

11.6 

3.4 

Measurement 

11 

7.0 

2.5 

Geometry 

6 

4.5 

1.3 

Graphing 

6 

4.2 

1.4 

Problem-Solving  Strategies 

3 

2.1 

0.9 

Cognitive  Level: 

Knowledge 

12 

8.7 

2.6 

Comprehension 

19 

12.9 

3.9 

Application/Problem  Solving 

24 

15.8 

4.9 
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Percentage  of  Students  Choosing  Each  Alternative 


Table  3-9  shows  the  percentage  of  students 
who  chose  each  alternative  (A,  B,  C,  and  D) 
for  each  multiple-choice  question.  The  correct 
response  (key)  for  each  question  is  also  identi- 
fied. The  results  shown  in  Table  3-9  can  best 
be  used  in  conjunction  with  the  parallel  tables 
in  the  jurisdiction  and  school  reports.  Varia- 
tions in  patterns  of  students’  responses  to 


questions  can  help  to  indicate  strengths  and 
weaknesses  in  local  educational  programs. 

Statistics  presented  in  this  table  are  based  on 
results  achieved  by  29  768  students  (those 
who  wrote  the  regular  form  and  those  who 
wrote  Form  B). 


Table  3-9 

Grade  6 Mathematics:  Booklets  1 and  2 
Provincial  Results  for  Individual  Multiple-Choice  Questions* 

June  1991 


Distribution  of  Distribution  of 

Responses  (%)  Responses  (%) 


Item 

Key 

A 

B 

C 

D 

Item 

Key 

A 

B 

c 

D 

1 

B 

6.7 

7.8 

5.3 

29  ; 

Iii!  • 

20.0 

48  <9 

7.1 

23,9 

2 

C 

3.1 

nil 

73.8 

17.6; 

30 

B 

4.9 

79,6 

11.3 

4,0 

3 

B 

16.8 

69,4 

10.0 

H 3-7; 

31  ! 

. AH 

61.0 

6.2 

4.8 

27*5 

4 

D 

14.0 

26:3 

7.0 

52,3 

32  1 

••■<£•••: 

8.5 

11,4 

65.7 

14.1 

5 

A 

63.8 

nil 

14.1 

iliii 

33  1 

A:-- 

62.6 

17,1 

11.8 

8,1 

6 

C 

2.4 

l3>9 

60.9 

iii 

34  | 

■111 

90.2 

4,5 

3.4 

Hi 

7 

A 

71.6 

lilli 

12.2 

3,6 

35  I 

■fm\ 

13.3 

22.7 

50.1 

13.5 

8 

A 

78.3 

ililll 

14.2 

: 44: 

36  | 

'A  • 

63.4 

18.3 

11.2 

6,6 

9 

! B 

1.4 

90,6 

5.7 

Mi 

37  i 

•A..; 

60.8 

■ill 

32.6 

2,6 

10 

! A 

78.5 

,7,4 

3.9 

•'  9.8 

38  I 

6.0 

13,5 

70.3 

10,0 

11 

! C 

7.7 

17*0 

47.1 

28-0 

39  I 

B 

4.5 

60.5 

31.0 

3,9 

12 

C 

8.0 

46 

82.6 

- 4,6; 

40  1 

:-c 

8.8 

7,6 

75.7 

7.6 

13 

! B 

6.3 

:746 

15.2 

•3,8 

41  1 

• •’  'b  ;-:  • • 

2.2 

69,5 

23.3 

4.8 

14 

III 

43.2 

2.1 

37.7 

16,7 

42  | 

. ‘ - ft  •’ : 

8.3 

7,9 

9.7 

73.7 

15 

D 

7.4 

9-6 

41.9 

41*0 

43  I 

■111! 

2.3 

13,5 

9.7 

74.2 

16 

D 

9.9 

nisi 

10.0 

69:6 

44  | 

‘•AH 

88.8 

5.7 

2.6 

2,5 

17 

! B 

9.4 

■ill 

10.6 

■III 

45  j 

• ::  ft  ! ' 

26.0 

4,7 

8.0 

61,2 

18 

B 

4.4 

76,0 

12.7 

"47 

46  ; 

vC  : ! 

2.7 

5,3 

86.8 

4.9 

19 

ID 

32.4 

■111 

6.0 

52.0 

47  i 

lllll 

8.0 

iliii 

11.2 

11,5 

20 

D 

2.5 

/"4T- 

5.4 

87,8 

48  1 

HbH 

7.8 

8,1 

79.0 

4.9 

21 

lllll 

4.5 

88.8 

3.1 

3,5 

49  ! 

•:  MU 

3.4 

4,7 

6.8 

84.6 

22 

lllll 

4.9 

37.5: 

8.2 

illi 

50  I 

ile. : 

13.1 

13,5 

60.5 

12.5 

23 

Bill! 

76.2 

15-0 

4.2 

43 

51  | 

B 

26.6 

iliii 

11.8 

1.1 

24 

B 

3.8 

63,1 

25.6 

7,'4 

52  I 

■ill 

6.1 

44.7 

16.9 

32.0 

25 

ft 

7.0 

• 12,4  • 

13.1 

67.4 

53  j 

8.3 

6,0 

61.6 

iliii 

26 

C 

3.9 

17,9 

73.9 

I*i:i 

54  j 

A 

58.5 

118,4 

14.4 

8.1 

27 

! & 

7.4 

6,2 

8.8 

77-5 : 

55  ! 

c. 

3.7 

ill!! 

71.2 

8,0 

28 

■ill 

4.5 

16.9 

50.7 

Mil 

*The  sum  of  the  percentages  for  each  question  may  be  less  than  100%  because  the  No  Response 
category  is  not  included.  This  category  is  less  than  1%  for  any  one  of  these  questions. 
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RESULTS  FOR  BOOKLET  3:  NUMBER  FACTS 


This  section  of  the  test  measured  student 
performance  in  operations  in  addition,  sub- 
traction, multiplication,  division,  and  mixed 
operations;  that  is,  it  measured  student  speed 
and  accuracy. 

Figure  3-3  and  Table  3-10  present  the  results 
in  terms  of  the  average  number  of  the  48 
questions  students  attempted  to  answer  in 


two  minutes  and  the  average  number  of 
questions  students  answered  correctly  in  two 
minutes. 

Statistics  presented  in  this  table  are  based  on 
results  achieved  by  31  930  students  (those 
who  wrote  the  regular  form  and  those  who 
participated  in  the  achievement-over-time 
study). 


Figure  3-3 


GRADE  6 MATHEMATICS 

RESULTS  FOR  NUMBER  FACTS:  SPEED  AND  ACCURACY  TESTS 

June  1991 


The  average  number  of 

222 

questions  attempted 

The  average  number  of  questions 
answered  correctly 


Table  3-10 

Grade  6 Mathematics 

Results  for  Number  Facts:  Speed  and  Accuracy  Tests 
June  1991 


Operations 

Time  Allotted 
in  Minutes 

Number  of 
Questions 

Average  Number 
Attempted 

Average  Number 
Correct 

Addition 

2 

48 

37.8 

37.0 

Subtraction 

2 

48 

35.6 

34.1 

Multiplication 

2 

48 

41.1 

39.7 

Division 

2 

48 

37.2 

34.7 

Mixed  Operations 

2 

48 

37.2 

34.2 
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GRADE  6 MATHEMATICS  EXAMINER’S  OBSERVATIONS 


The  following  is  a discussion  of  specific  areas  of  strength  and  of  difficulty  for  Grade  6 students. 
Sample  questions  are  provided  to  highlight  the  strengths  and  weaknesses  of  students  achiev- 
ing at  the  acceptable  standard  and  at  the  standard  of  excellence.  For  each  sample  question, 
the  correct  answer  is  marked  with  an  asterisk.  The  percentage  of  students  choosing  each 
alternative  is  also  provided. 


Acceptable  Standard — Sample  Questions  and  Commentary 


Use  the  chart  below  to  answer 
question  21. 


EAGLETON  SCHOOL  DISTRICT  ENROLMENT 

School 

1 

©@©©©(3 

2 

©©©©© 

3 

©©©© 

©-too***  j 

21.  What  is  the  TOTAL  enrolment  of 
ALL  the  schools  in  the  Eagleton 
School  District? 


Generally,  the  Grade  6 teachers  who  analysed  the 
standards  built  into  the  test  felt  that  the  test  re- 
flected the  nature  of  the  Grade  6 Mathematics  pro- 
gram. As  well,  these  teachers  felt  that  the  provision 
of  a section  in  which  calculators  were  permitted  was 
consistent  with  the  NCTM  Standards  and  with  the 
philosophy  and  rationale  of  the  role  of  technology  in 
the  Elementary  Mathematics  Curriculum  Guide, 
1982.  The  teachers  also  felt  that  because  the  pro- 
gram has  been  in  operation  for  almost  10  years,  the 
test  items  were  predictable;  in  other  words,  there 
were  no  surprises  in  the  content  of  the  test. 


4.5  A.  1725  students 
88.8  *B.  1750  students 

3.1  C.  1775  students 

3.5  D.  1800  students 

Question  21  required  students  to  read 
a simple  chart  and  then  apply  a one-step 
problem-solving  strategy  to  a real-life 
situation. 


General  Statement  on  Student  Performance 

Overall,  students  were  able  to  solve  one-step  prob- 
lems, complete  routine  operations,  and  apply  their 
knowledge  to  straightforward  problem-solving  ques- 
tions. As  well,  students  showed  a knowledge  of  and 
an  ability  to  use  basic  measurement  skills. 


12.  Which  group  of  symbols  represents 
standard  units  of  measure  from 
SMALLEST  to  LARGEST? 


Listed  below  are  some  areas  where  students  per- 
forming at  the  acceptable  standard  and  beyond 
showed  a strong  grasp  of  mathematics. 


8.0  A.  dm,  cm,  m 
4.6  B.  hm,  km,  mm 
82.6*C.  cm,  m,  km 
4.6  D.  mm,  km,  m 

Question  12  required  students  to  recognize 
units  of  measure  and  to  apply  a routine 
procedure. 


Grade  6 students  could  do  the  following: 

• read  simple  charts  and  graphs  (see  question  21) 

• recognize  place  value,  parallel  lines,  congruency, 
problem-solving  stages,  co-ordinates,  ordered  pairs, 
and  units  of  measure  and  equivalencies  (see 
question  12) 

• use  routine  procedures  and  operations  in  addition, 
subtraction,  multiplication,  division,  and  estima- 
tion (see  question  46) 

• apply  simple  one-step  problem-solving  strategies  to 
real-life  situations  (see  question  21). 
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46.  9 )4321  equals 

2.7  A.  318  R1 
5.3  B.  329  R2 
86.8*C.  480  R1 
4.9  D.  491  R2 

Question  46  required  students  to  perform 
a routine  procedure  in  division. 


However,  students  could  not  do  the  following  well 
enough: 

• complete  multistep  problems 

• transfer  knowledge  to  new  or  real-life  problem- 
solving situations 

• understand  which  operations  to  use  when  solving 
problems. 


Standard  of  Excellence — Sample  Questions  and  Commentary 


Use  the  graph  below  to  answer  question  24. 


24.  The  graph  shows  the  attendance  at 
the  Calgary  Stampede  for  one  week. 
Saturday’s  attendance  was  DOUBLE 
that  of  Sunday’s.  Which  circle  should 
be  filled  in  to  show  Saturday’s  atten- 
dance? 

3.8  A.  Z 
63.1  *B.  Y 
25.6  C.  X 
7.4  D.  V 

Question  24  required  students  to  read  and 
interpret  a graph,  and  to  apply  a multistep 
problem-solving  strategy  to  a real-life 
situation. 


11.  1 m + 4 cm  + 2 mm  equals 

7.7  A.  10  402  mm 

17.0  B.  1402  mm 
47.1*C.  1042  mm 

28.0  D.  142  mm 

Question  11  required  students  to  manipulate  mixed  units. 
Use  the  information  below  to  answer  question  53. 


53.  At  this  rate,  how  long  will  it  take  the  skier  to  go  15  km? 

8.3  A.  150  min 
6.0  B.  90  min 
61.6*C.  75  min 
23.9  D.  45  min 

Question  53  required  students  to  apply  ratios  to  a multistep 
problem  using  a real-life  situation. 


Students  who  achieved  at  the  standard  of  excellence  demonstrated  more  success  and  under- 
standing in  solving  symbolic  and  multistep  problems  than  other  students  did.  Specifically, 
those  students  achieving  at  the  standard  of  excellence  could  do  the  following: 

• demonstrate  an  understanding  of  equivalent  fractions  and  place  value  in  a problem-solving 
setting 

• recognize  flips,  slides,  and  cylinder  construction 

• read  and  interpret  graphs  in  problem-solving  situations  (see  question  24) 
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• manipulate  mixed  units,  arrange  integers,  calculate  averages,  add  decimals,  convert  frac- 
tions to  decimals,  apply  ratios,  and  find  area,  volume,  and  perimeter  (see  questions  11  and 
53) 

• apply  more  complex  multistep  problem-solving  strategies  to  real-life  situations  (see 
questions  24  and  53). 

These  students  generally  showed  strengths  in  all  areas  that  were  assessed. 


New  Features  of  the  Test 

The  introduction  of  a calculator  and  of  the  “cluster”  questions  on  the  June  1991  Grade  6 Math- 
ematics Achievement  Test  was  well  accepted  by  most  teachers.  Teachers  remarked  that  the 
test  could  have  been  done  without  using  a calculator;  therefore,  students  without  calculators 
were  not  disadvantaged.  Teachers  noted  that  more  than  adequate  time  was  provided  for  stu- 
dents to  complete  Booklet  1 and  Booklet  2. 


Issues 

Some  teachers  and  students  felt  that  not  enough  time  was  provided  to  complete  the  five  timed 
tests.  These  tests  are  designed  to  test  only  speed  and  accuracy  with  number  facts.  To  do  this, 
a time  element  is  built  in  to  the  tests  to  discriminate  between  students  who  know  the  number 
facts  and  those  who  do  not. 


Concluding  Comments 

Overall,  the  results  showed  that  not  enough  students  performed  at  the  acceptable  standard 
and  that  a larger  number  of  students  than  expected  performed  at  the  standard  of  excellence. 

In  other  words,  although  some  students  are  achieving  very  well,  most  students  are  not  learn- 
ing mathematical  concepts  and  skills  well  enough.  From  a careful  consideration  of  the  ques- 
tions that  students  found  difficult,  it  seems  that  students  lack  understanding  in  some  of  the 
concepts  and  their  applications  to  real-life  situations.  Students  need  more  work  and  time  with 
manipulatives  at  the  concrete  level  to  help  them  understand  abstract  concepts.  Further,  an 
increased  emphasis  on  problem  solving  in  a variety  of  contexts  would  help  students  connect 
their  newly  acquired  mathematical  concepts  to  real-life  situations. 
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SECTION  4 

GRADE  9 SOCIAL  STUDIES 


GENERAL  DESCRIPTION 


The  Grade  9 Social  Studies  Achievement  Test 
had  two  parts.  Part  A consisted  of  60 
multiple-choice  questions  worth  70%  of  the 
total  test  mark.  These  questions  assessed 
students’  knowledge  as  well  as  their 
understanding  and  application  of  process 
skills. 

Part  B consisted  of  one  written-response 
question  worth  30%  of  the  total  mark.  Papers 
were  scored  on  two  dimensions:  Support  of 
Position,  worth  20%,  and  Expression,  worth 
10%. 

The  total  writing  time  for  the  test  was  1 3/4 
hours.  The  questions  on  the  test  were 
organized  into  two  components: 

• Knowledge  (facts,  concepts,  generalizations) 

• Process  Skills 


Statistics  for  total  test  and  for  components 
were  based  on  the  results  achieved  by  27  646 
students  (24  720  wrote  the  regular  form,  and 
2 926  participated  in  the  achievement-over- 
time study,  as  shown  in  Table  4-3).  This 
section  of  the  report  answers  the  following 
questions: 

• How  many  Grade  9 students  wrote  each 
form  of  the  test  or  were  absent  and  exempt? 

• What  percentage  of  Grade  9 students 
attained  the  acceptable  standard  in  Social 
Studies  according  to  provincial  criteria? 

• What  percentage  of  Grade  9 students 
attained  the  standard  of  excellence  in 
Social  Studies  according  to  provincial 
criteria? 

• What  did  Grade  9 students  know  and  what 
could  they  do  in  Social  Studies? 

• What  parts  of  the  Social  Studies  curriculum 
caused  Grade  9 students  difficulty? 


SUMMARY  OF  RESULTS 


Results  In  Relation  to  Standards 

Results  show  that  67.9%  of  students  who  wrote  the  test  achieved  the  acceptable  standards  on 
both  components  and  5.1%  achieved  the  standards  of  excellence  on  both  components.  The 
results  were  lower  than  expected. 

Average  Score 

The  average  total  test  score  was  60.5%  with  a standard  deviation  of  15.2. 


CONTENT  OF  THE  TEST 


The  test  was  based  on  the  1989  Alberta  Social 
Studies  Program  of  Studies.  The  test 
measured  two  major  components: 

• Knowledge  (facts,  concepts,  generalizations) 

• Process  Skills 

All  questions  were  drawn  from  the  content  of 
the  three  topics  prescribed  for  Grade  9: 

• Topic  A:  Economic  Growth:  U.S.A. 

• Topic  B:  Economic  Growth:  U.S.S.R. 

• Topic  C:  Canada:  Responding  to  Change 


The  multiple-choice  section  of  the  test  covered 
all  three  topics  and  had  33  per  cent  emphasis 
on  the  knowledge  component  and  37  per  cent 
emphasis  on  the  process  skills  component. 

The  written-response  portion  assessed 
students’  communication  skills.  Therefore, 
the  total  emphasis  was  on  process  skills.  The 
assignment  presented  to  students  related  to 
Topic  C:  Canada:  Responding  to  Change. 


-33 


ACHIEVEMENT  TEST  BLUEPRINT 


The  test  blueprint  for  Part  A:  Multiple  Choice  shows  the  distribution  of  questions  according  to 
the  curricular  content  area  (topic)  being  assessed  and  according  to  the  knowledge  and  process 
skill  objectives  called  for  by  the  question. 

Table  4-1 

Grade  9 Social  Studies 
Achievement  Test  Blueprint 
Part  A:  Multiple  Choice 
June  1991 


Topic/ 

Concept 

Reporting 

Categories 

Topic  A 

Economic  Growth: 

U.S.A. 

Topic  B 

Economic  Growth:  U.S.S.R. 

Topic  C 

Canada:  Responding  To 
Change 

Propor- 
tion of 
Total 

Objective 

Reporting 

Categories 

Industri- 

alization 

Market 

Economy 

Quality 
of  Life 

Geog- 

raphy 

Industri- 

alization 

Centrally 

Planned 

Economy 

Quality 
of  Life 

Tech- 

nology 

Mixed 

Economy 

Quality 
of  Life 

Score 

Knowledge 

Objectives 

Under- 

stands 

general- 

izations, 

concepts, 

related 

concepts, 

terms, 

and 

facts 

1,  2,  3, 
19 

10,  12,  17 

13, 15, 

25 

22,  23, 
24,  32 

29,  30 

36,  37, 
38 

44,  49, 
50,51 

43,  46, 
52 

55,  58 

33% 

a M 
IS  « 
00  > 

Locating, 
interpret- 
ing, and 
organiz- 
ing 

7 

4,  6, 16 

20 

21 

40 

27,  28 

33 

41,42 

54,  56,  57, 
60 

18.5% 

£ o 
<3  u 

1° 

Analysing, 
synthesiz- 
ing, and 
evaluat- 
ing 

8,9 

5,  11,  18 

14 

26,31 

34,  35 

39 

47,48, 

53 

59 

45 

18.5% 

Proportion 

of 

8% 

10% 

5% 

2% 

8% 

7% 

6% 

11% 

10% 

3% 

70% 

Total 

Score 

23.3% 

23.3  % 

23.3% 
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The  test  blueprint  for  Part  B:  Written  Response  shows  the  writing  assignment  presented  to 
students  and  the  emphasis  of  communication  that  was  assessed. 


Table  4-2 

Grade  9 Social  Studies 
Achievement  Test  Blueprint 
Part  B:  Written  Response 
June  1991 


Writing  Assignment 

Reporting  Category 

Proportion 
of  Score 

Recycling  is  a growing  concern  in  Alberta. 

1.  Support  of  Position: 

Students  are  expected  to 

20% 

HOW  SHOULD  THE  GOVERNMENT 

support  their  position 

OF  ALBERTA  BE  INVOLVED  IN 

by  organizing  their  points 

RECYCLING  PROGRAMS? 

of  argument  convincingly, 
and  by  using  specific  and 

In  presenting  your  opinion,  consider  economic 

pertinent  examples  and 

and  environmental  factors. 

Your  written  response  may  take  the  form 

details  to  illustrate  their 
points  of  argument. 

of  a letter  to  the  government,  a speech  to 

2.  Expression: 

10% 

your  class,  a newspaper  editorial,  or  a 

Students  are  expected  to 

position  paper. 

express  their  ideas  and 
arguments  with  clear, 

To  support  your  opinion,  use  examples  and 

specific,  accurate  language, 

evidence  from  the  Information  to  Consider 

and  to  use  correct  conven- 

on page  25  of  the  test  booklet,  as  well  as  from 

tions  of  spelling,  punctuation, 

YOUR  KNOWLEDGE  OF  ECONOMIC 
SYSTEMS. 

and  grammar. 
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STUDENT  PARTICIPATION 


In  June  1991,  principals  reported  a total 
population  of  32  553  students  in  Grade  9. 
Table  4-3  presents  the  number  and  percent- 
age distribution  of  students  who  wrote  the 
Grade  9 Social  Studies  Achievement  Test, 


who  were  absent,  and  who  were  exempted. 
Results  for  students  in  French  Immersion  or 
Francophone  programs  are  reported  sepa- 
rately to  participating  schools  and  jurisdic- 
tions. 


Table  4-3 

Grade  9 Social  Studies 
Student  Participation 
June  1991 


Category 

Number 
of  Students 

Percentage 
of  Students 

Total  Number  of  Students  in  J^gtftar ' . f .i||l 

27646 

84.9 

Students  Who  Wrote  the  Regular  Form** 

24  720 

• ' Students  Who  Wrdtetbie  AchibVdmMt-(^  Forms: 

!'  2926 

' V:  90  " 

Form  R{Red — Same  as  the  Regular  Form 

$89 

3M 

Form  T (Turquoise) 

973 

3~Q 

/:  ,•  Form  C (Charcoal)  . • 

\ • 964 

3M 

Other  Students  Who  Wrote:*** 

1573 

4.8 

Francophone  Students — Wrote  in  French 

152 

0.5 

Francophone  Students — Wrote  in  English 

0 

0.0 

French  Immersion  Students — Wrote  in  French 

904 

2.8 

French  Immersion  Students — Wrote  in  English 

0 

0.0 

Language-of-Testing  Students — Wrote  in  French 

258 

0.8 

Language-of-Testing  Students — Wrote  in  English 

259 

0.8 

Students  Absent 

1 647 

5.1 

Students  Exempted  from  Writing 

1 687 

5.2 

Categories  of  Exemption: 

1.  Special  Needs  Students 

796 

2.4 

2.  Subject  Was  Not  Taught  This  Term 

454 

1.4 

3.  ESL  Students 

352 

1.1 

4.  Language  of  Instruction  Was  Not  English 

0 

0.0 

5.  Other  (as  approved  by  the  Superintendent) 

85 

0.3 

Total  Principals’  Reported  Population: 

Test  Day,  June  1991 

32  553 

100.0 

Grade  9 Enrolment:  September  30, 1990 

35  512 

* Provincial  results  are  based  on  test  scores  achieved  by  students  who  were  in  the  “regular”  (Mathematics  or  Science 
or  Social  Studies)  programs.  Such  students  are  defined  as  those  who  were  instructed  in  English  and  wrote  the 
English  form  of  the  achievement  test,  and  those  students  who  were  instructed  in  a language  other  than  English 
or  French  and  wrote  the  English  form  of  the  test.  These  students  are  indicated  by  the  shaded  area  of  the  table. 

**  Regular  Form  refers  to  the  unmodified  1991  test.  The  modified  tests  are  described  in  Section  8 of  this  report. 

***Results  for  Francophone  or  French  Immersion  students  or  for  students  participating  in  the  language-of-testing 
study  are  reported  directly  to  participating  schools  and  jurisdictions  and  are  not  calculated  in  the  provincial 
results. 
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RESULTS  IN  RELATION  TO  STANDARDS 


Figures  4-1  and  4-2  and  Table  4-4  show  the  percentage  of  students  achieving  the  acceptable 
standards  and  the  standards  of  excellence  on  components  of  the  test.  The  levels  of  performance 
were  lower  than  expected  for  the  acceptable  standards  as  well  as  for  the  standards  of  excel- 
lence. 


Figure  4-1 


GRADE  9 SOCIAL  STUDIES 

PERCENTAGE  OF  STUDENTS  ACHIEVING  ACCEPTABLE 
STANDARDS  OR  HIGHER  ON  COMPONENTS  OF  THE  TEST 

June  1991 


^ Achievement  Standards*  |iTj  Actual  Results5 

(See  Appendix  A) 


*the  percentage  of  students  in  the  province  expected  to  meet  the  acceptable  standards  or  higher 
**the  percentage  of  students  in  the  province  who  met  the  acceptable  standards  or  higher 


Figure  4-2 

GRADE  9 SOCIAL  STUDIES 

PERCENTAGE  OF  STUDENTS  ACHIEVING  STANDARDS  OF 
EXCELLENCE  ON  COMPONENTS  OF  THE  TEST 


Achievement  Standards* 
(See  Appendix  A) 


II  [Actual  Results** 


*the  percentage  of  students  in  the  province  expected  to  meet  the  standards  of  excellence 

**the  percentage  of  students  in  the  province  who  met  the  standards  of  excellence 
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Table  4-4 

Grade  9 Social  Studies 

Students  Achieving  Standards  on  Components  of  the  Test 

June  1991 


Reporting  Category 

Maximum 

Possible 

Score 

Provincial 
Assessment 
Standard* 
(Raw  Score) 

Provincial 
Achievement 
Standard** 
(Per  Cent) 

Expected 

Number 

Actual 

Number 

Actual 
Per  Cent 

Standard  of  Excellence 

Both  Major  Components*** 

33  + 67 

29  + 53 

10 

2 765 

1402 

5.1 

Facts,  Concepts, 
Generalizations  Only 

33 

29 

15 

4 147 

3 081 

11.1 

Process  Skills  Only 

67 

53 

15 

4 147 

2 711 

9.8 

Acceptable  Standard 
or  Higher 

Both  Major  Components*** 

33  + 67 

17  + 31 

80 

22  117 

18  774 

67.9 

Facts,  Concepts, 
Generalizations  Only 

33 

17 

85 

23  499 

20  027 

72.4 

Process  Skills  Only 

67 

31 

85 

23  499 

22  495 

81.4 

Below  Acceptable  Standard 

N/A 

N/A 

N/A 

N/A 

3 898 

14.1 

on  Both  Components 

*The  Provincial  Assessment  Standard  is  a score  determined  by  appropriate  standard-setting  procedures  and  is 
the  lowest  score  students  must  achieve  for  their  performance  to  be  judged  “acceptable”  and/or  “excellent”  in 
relation  to  curricular  expectations.  See  Appendix  A. 


**The  Provincial  Achievement  Standard  refers  to  the  percentage  of  students  expected  to  meet  or  exceed  the 
Provincial  Assessment  Standard.  See  Appendix  A. 

***In  previous  years,  a standard  for  the  total  test  was  reported.  An  individual  could  meet  the  standard  for  the  test 
overall  but  might  not  achieve  the  standard  for  both  components  of  the  test.  The  new  method  of  reporting  allows 
for  a more  meaningful  interpretation  of  the  results.  To  allow  comparison  with  results  from  previous  years,  the 
total  test  score  is  reported  in  Table  4-5. 
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Table  4-5  shows  the  percentage  of  students  achieving  the  acceptable  standard  and  the  stand- 
ard of  excellence  on  the  total  test.  These  levels  of  performance  were  lower  than  the  expecta- 
tions for  both  standards. 

Table  4-5 

Grade  9 Social  Studies 

Students  Achieving  Standards  on  the  Total  Test 
June  1991 


Reporting  Category 

Maximum 

Possible 

Score 

Provincial 
Assessment 
Standard* 
(Raw  Score) 

Provincial 
Achievement 
Standard** 
(Per  Cent) 

Expected 

Number 

Actual 

Number 

Actual 
Per  Cent 

Total  Test: 

Standard  of  Excellence 

100 

82 

15 

4 147 

2 242 

8.1 

Acceptable  Standard 
or  Higher 

100 

48 

85 

23  499 

21  809 

78.9 

*The  Provincial  Assessment  Standard  is  a score  determined  by  appropriate  standard-setting  procedures  and  is  the 
lowest  score  students  must  achieve  for  their  performance  to  be  judged  “acceptable”  and/or  “excellent”  in  relation 
to  curricular  expectations.  See  Appendix  A. 


**The  Provincial  Achievement  Standard  refers  to  the  percentage  of  students  expected  to  meet  or  exceed  the  Provin- 
cial Assessment  Standards.  See  Appendix  A. 


It  should  be  noted  that  the  actual  percentages 
for  Grade  9 Social  Studies  (8.1  and  78.9)  are 
based  on  the  27  646  students  in  the  regular 
programs  who  wrote  the  test. 

If,  however,  the  percentages  are  based  on  the 
total  population  reported  by  principals 
(32  553),  less  the  French  Immersion  and 
Francophone  students  who  wrote  the  test 
(1  573),  the  percentages  would  be: 

7.2%  achieving  the  standard  of  excellence 
70.4%  achieving  the  acceptable  standard 

If  the  percentages  are  based  on  the  Septem- 
ber 30,  1990,  Grade  9 enrolment  (35  512),  less 
the  French  Immersion  and  Francophone 
students  who  wrote  the  test  (1  573),  the 
percentages  would  be: 

6.6%  achieving  the  standard  of  excellence 
64.3%  achieving  the  acceptable  standard 

It  is  emphasized  that  the  above  percentages, 
based  on  total  population  and  enrolment 
figures,  present  the  worst  possible  scenario. 


It  is  highly  unlikely  that  none  of  the  students 
who  was  absent,  exempt,  or  not  accounted  for 
would  have  achieved  standards.  The  absence 
of  information  on  these  students  is  nonethe- 
less problematic. 

The  number  of  students  achieving  the  accept- 
able standard  and  the  standard  of  excellence 
for  each  jurisdiction  was  analysed  to  deter- 
mine whether  jurisdictions  were  below,  meet- 
ing, or  exceeding  provincial  achievement 
standards.  Jurisdictions  classified  as  meeting 
provincial  achievement  standards  were  those 
for  which  the  difference  between  the  actual 
number  of  students  and  the  expected  number 
of  students  at  or  above  standards  was  not 
statistically  significant.  A 95%  confidence 
interval  was  used;  this  criterion  means  that 
differences  are  only  reported  when  there  is  a 
5%  or  smaller  probability  that  a difference  of 
that  size  could  occur  by  chance.  The  results 
are  reported  in  Tables  4-6  and  4-7.  The 
percentages  in  the  tables  are  based  on  200 
jurisdictions  (including  private  schools). 
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Table  4-6 

Grade  9 Social  Studies 

Percentage  Distribution  of  Jurisdictions*  Meeting 
Provincial  Achievement  Standards  on  Components  of  the  Test 

June  1991 


Component 

Jurisdictions 
Not  Meeting 
Provincial 
Achievement 
Standard 
(Per  Cent) 

Jurisdictions 
Meeting  Prov- 
incial Achieve- 
ment Standard 
(Per  Cent) 

Jurisdictions 

Exceeding 

Provincial 

Achievement 

Standard 

(Per  Cent) 

Standard  of  Excellence 

Both  Major  Components 

32.0 

66.3 

1.7 

Facts,  Concepts,  and  Generalizations  Only 

27.5 

68.6 

3.9 

Process  Skills  Only 

30.3 

67.5 

2.2 

Acceptable  Standard  or  Higher 

Both  Major  Components 

59.5 

38.8 

1.7 

Facts,  Concepts,  and  Generalizations  Only 

63.4 

36.0 

0.6 

Process  Skills  Only 

29.8 

68.0 

2.2 

Table  4-7 

Grade  9 Social  Studies 

Percentage  Distribution  of  Jurisdictions*  Meeting 
Provincial  Achievement  Standards  on  the  Total  Test 
June  1991 


Reporting  Category 

Jurisdictions 
Not  Meeting 
Provincial 
Achievement 
Standard 
(Per  Cent) 

Jurisdictions 
Meeting  Prov- 
incial Achieve- 
ment Standard 
(Per  Cent) 

Jurisdictions 

Exceeding 

Provincial 

Achievement 

Standard 

(Per  Cent) 

Total  Test: 

Standard  of  Excellence 

39.3 

59.0 

1.7 

Acceptable  Standard  or  Higher 

39.3 

60.1 

0.6 

*Jurisdictions  with  fewer  than  six  students  are  excluded,  as  the  statistical  significance  of  the  difference  between  the 
number  actually  meeting  the  standard  and  the  number  expected  to  meet  the  standard  when  calculated  and  reported 
is  not  educationally  meaningful. 
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RESULTS  FOR  PART  A:  MULTIPLE  CHOICE 


Reporting  Categories 


Table  4-8  shows  the  total  marks  possible  and 
the  provincial  raw  score  results  for  the  report- 
ing categories  of  the  multiple-choice  portion  of 
the  Grade  9 Social  Studies  Achievement  Test. 

It  is  important  to  stress  that  the  averages  on 
the  various  reporting  categories  cannot  be 
directly  compared  with  one  another.  Rather, 
the  results  shown  in  Table  4-8  can  best  be 
used  in  conjunction  with  parallel  tables  in  the 


jurisdiction  and  school  reports.  Variations  in 
patterns  of  students’  responses  to  questions 
can  help  to  indicate  strengths  and  weaknesses 
in  local  educational  programs. 

Statistics  presented  in  this  table  are  based  on 
results  achieved  by  25  709  students  (those 
who  wrote  the  regular  form  and  those  who 
wrote  Form  R). 


Table  4-8 

Grade  9 Social  Studies 
Raw  Score  Results  by  Reporting  Category 
Part  A:  Multiple  Choice 

June  1991 


Reporting  Category 

Number 

of 

Questions 

Raw  Score 
Average 

Raw  Score 
Standard  Deviation 

Total 

60 

37.4 

10.4 

Topic  A:  Economic  Growth:  U.S.A. 

20 

12.6 

3.8 

Topic  B:  Economic  Growth:  U.S.S.R. 

20 

11.8 

3.6 

Topic  C:  Canada:  Responding  to  Change 

20 

13.0 

4.1 

Recall  of  Knowledge:  Facts,  Concepts, 

and  Generalizations 

28 

17.9 

5.3 

Application  of  Process  Skills 

32 

19.4 

5.6 

Recall  of  Knowledge: 

Topic  A 

9 

5.7 

2.0 

Topic  B 

10 

6.3 

2.2 

Topic  C 

9 

6.0 

2.1 

Application  of  Process  Skills: 

Locating,  Interpreting,  and  Organizing 

16 

9.4 

3.0 

Analysing,  Synthesizing,  and  Evaluating 

16 

10.0 

3.2 
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Percentage  of  Students  Choosing  Each  Alternative 


Table  4-9  shows  the  percentage  of  students 
who  chose  each  alternative  (A,  B,  C,  and  D) 
for  each  multiple-choice  question.  The  correct 
response  (key)  for  each  question  is  also  identi- 
fied. 

The  results  shown  in  Table  4-9  can  best  be 
used  in  conjunction  with  the  parallel  tables  in 


the  jurisdiction  and  school  reports.  Variations 
in  patterns  of  students’  responses  to  questions 
can  help  to  indicate  strengths  and  weaknesses 
in  local  educational  programs. 

Statistics  presented  in  Table  4-9  are  based  on 
results  achieved  by  25  709  students  (those 
who  wrote  the  regular  form  and  those  who 
wrote  Form  R). 


Table  4-9 

Grade  9 Social  Studies 

Provincial  Results  for  Individual  Multiple-Choice  Questions* 

June  1991 


Item 

Key 

Distribution  of  Responses 

% 

Item 

Key 

Distribution  of  Responses 

% 

A 

B 

c 

D 

A 

B 

c 

D 

1 

79.0 

: 13A 

2.9 

: ::-  4.T 

31 

6.5 

1:  9 A 

26.8 

57.1 

2 

iiiii 

9.8 

439 

16.9 

294: 

32 

B 

12.2 

;im 

5.2 

12.5 

3 

D 

14.7 

22.8  : 

3.4 

59  0 

33 

;l  D ' 

15.8 

:•  3:4: 

3.9 

76.8 

4 

' -A" 

85.4 

lAil 

6.4 

4.0 

34 

20.1 

15.5 

51.6 

12,7 

5 

B 

27.0 

ms 

6.3 

BS 

35 

S:A  V 

52.1 

17  1 

20.9 

f 9.8 

6 

llliil 

13.8 

ms;: 

21.8 

565 

36 

>VA::J 

53.4 

MS 

8.6 

21.9 

7 

V:'£-4 

16.9 

Hi® 

65.4 

135 

37 

D V 

11.5 

14.1 

20.2 

54.0 

8 

•ilii 

65.9 

i$;3.:. 

17.4 

38 

V:-  B ; 

5.9 

77.6 

4.9 

9 

B 

35.1 

53.4  : 

5.0 

ss 

39 

D 

14.6 

17,6 

14.8 

52,9 

10 

nisi 

10.9 

10.3  j 

28.1 

60.3 

40 

-•VVA:  : 

32.0 

23.3 

29.8 

14:7 

11 

IB:- 

22.4 

62.1 

6.5 

8.9 

41 

V : vbi ; 

27.9 

1 5.9' 

9.4 

56.6 

12 

Hi  ‘ 

8.7 

541 : 

28.8 

: 8.1 

42 

llill 

28.8 

: 134: 

25.3 

32.4 

13 

e • 

16.6 

61.8 

l'7,o. 

43 

IIC-I 

15.2 

52.9 

10,0 

14 

llill 

11.2 

:t4:6i 

28.2 

55.9 

44 

B II 

5.0 

: 664: 

10.6 

284 

15 

B 

14.1 

66.  9: 

8.4 

10/4 

45 

I'A 

60.4 

244 

8.1 

16 

Slv; 

49.9 

i2;o 

26.0 

4Ml 

46 

••  •£/..* 

13.9 

53.1 

18.8 

144 

17 

'• B ' 

5.6 

■Ms:: 

11.8 

105 

47 

wm 

7.1 

104 

78.0 

! '4.Y- 

18 

lA. 

66.1 

15.2 

110.7: 

48 

:.;:'cvr 

7.6 

77.3 

7;  2'' 

19 

icv 

11.2 

wm 

79.3 

•i-25  • 

49 

;Sb  ' 

11.7 

78.2 

4.7 

5>2 

20 

A • 

72.1 

SA  ■ 

12.2 

5.3 

50 

B 

5.5 

66;o 

15.2 

lliilf 

21 

'••••  M 

13.2 

10.7 

64.2 

11,7 

51 

llill 

73.6 

8.5 

4.6 

13.2 

22 

B 

3.1 

69.7V 

11.3 

46:9: 

52 

D 

8.4 

14.7 

23.5 

53.3 

23 

A 

49.0 

ite 

30.9 

10.2 

53 

C 

3.5 

•ii6i.; 

83.9 

.’6, 2’ 

24 

l'6l 

9.1 

85 

69.2 

13.1 ; 

54 

B • 

6.7 

645 

13.2 

ISA 

25 

B 

11.6 

535 

14.3 

205 

55 

A 

89.8 

2,5 

3.7 

••13,9' 

26 

20.6 

::V8.4.;. 

16.5 

54.3; 

56 

•IA':| 

52.8 

11.2 

19.6 

' 16;2: 

27 

• ••  B 

12.3 

9.5 

285 

57 

19.5 

33.6 

6.5 

40,2 

28 

M CT" 

11.3 

Ma- 

67.1 

17.1 

58 

c 

4.0 

1 14:5- 

78.3 

29 

l&l: 

47.9 

M13.9- 

28.5 

95: 

59 

• B " 

7.8 

#4 

14.8 

. ■■■  7i2.: 

30 

: • 

81.5 

8.2 

35 

60 

V A • 

78.3 

1: 4;s 

5.9 

10.9 

*The  sum  of  the  percentages  for  each  question  may  be  less  than  100%  because  the  No  Response 
category  is  not  included.  This  category  is  less  than  1%  for  any  one  of  these  questions. 
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RESULTS  FOR  PART  B:  WRITTEN  RESPONSE 


Results  for  Part  B:  Written  Response  are  most 
clearly  understood  in  the  context  of  the  as- 
signment students  responded  to  and  in  the 
context  of  the  scoring  guides.  The  average 
raw  score  for  written  response  was  17.0  out  of 
a possible  30,  with  a standard  deviation  of  5.2. 
Complete  scoring  guides  are  available  from 


the  Student  Evaluation  Branch,  Alberta 
Education. 

All  schools  should  have  extra  copies  of  the 
Part  B:  Written  Response  test  to  use  in  con- 
junction with  information  provided  in  this 
Provincial  Report. 


Scoring  Reliability 


Although  the  papers  were  scored  on  a one- 
marker  system,  192  randomly  selected  papers 
were  re-marked  so  that  a second  set  of  scores 
was  available  to  confirm  scoring  consistency. 
Of  the  scores  awarded  on  the  second  reading, 
93.5%  were  identical  to  the  original  score  on 
the  same  scale  or  varied  by  only  one  point.  It 
is  important  to  note  that  the  one-marker 
system  produces  results  that  are  reliable  for 
groups  of  25  or  more  students.  Achievement 
test  scores,  however,  are  less  reliable  for 
individual  students. 

The  results  presented  in  Table  4-10  are  best 
considered  in  terms  of  the  percentage  of 


students  that  markers  judged  to  have  pre- 
sented work  that  was  3 (Acceptable)  or  higher 
for  any  reporting  category.  It  is  possible  to 
draw  conclusions  about  local  program 
strengths  and  weaknesses  by  comparing  local 
percentages  of  3 (Acceptable)  or  higher  scores 
on  each  reporting  category  with  the  provincial 
percentages. 

Statistics  presented  in  Table  4-10  are  based 
on  results  achieved  by  25  709  students  (those 
who  wrote  the  regular  form  and  those  who 
wrote  Form  R). 


Table  4-10 

Grade  9 Social  Studies 
Part  B:  Written  Response 
Percentage  Distribution  of  Written-Response 
Scale  Points  by  Scoring  Category 
June  1991 


Scale  Point 

Support  of  Position 

Expression 

5 (Excellent) 

3.8 

3.7 

4 (Good) 

15.9 

19.1 

3 (Acceptable) 

42.6 

52.6 

2 (Limited) 

29.0 

20.3 

1 (Poor) 

7.8 

3.4 

Ins  (Insufficient  or 
No  Response) 

1.0 

1.0 
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GRADE  9 SOCIAL  STUDIES  EXAMINER’S  OBSERVATIONS 


Multiple  Choice 

Students  found  the  multiple-choice  section  of  the  test  very  challenging.  The  following  is  a 
discussion  of  specific  areas  of  strength  and  of  difficulty  for  Grade  9 students.  Sample  questions 
are  provided  to  highlight  the  strengths  and  weaknesses  of  students  achieving  at  the  acceptable 
standard  and  at  the  standard  of  excellence.  For  each  sample  question,  the  correct  answer  is 
marked  with  an  asterisk.  The  percentage  of  students  choosing  each  alternative  is  also  pro- 
vided. 


Acceptable  Standard — Sample  Questions  and  Commentary 


38.  The  most  general  definition  of  quality  of  life  is 

5.9  A.  your  job  satisfaction 

77.6  *B.  how  happy  you  are 

4.9  C.  your  level  of  education 

11.6  D.  how  much  money  you  have 

Question  38  required  students  to  have  a basic  understanding  of  quality  of 
life.  Students  achieving  at  the  acceptable  standard  have  a good  understanding 
of  this  concept.  Few  students  confused  quality  of  life  with  materialism. 


31.  President  Gorbachev’s  attempted  reforms  MOST  CLOSELY  parallel 

6.5  A.  Lenin’s  War  Communism 

9.4  B.  Stalin’s  collectivization 
26.8  C.  Stalin’s  five-year  plans 
57.1  *D.  Lenin’s  New  Economic  Policy 

Question  31  required  students  to  apply  their  knowledge  of  historical  events  to 
current  reforms  in  the  Soviet  Union.  Results  indicate  that  students  achieving  at  the 
acceptable  standard  were  unable  to  apply  their  knowledge  of  economic  planning  in 
this  question;  however,  students  achieving  at  the  standard  of  excellence  were  clearly 
able  to  do  so. 


The  strengths  of  students  who  demonstrated  an  acceptable  standard  of  performance  include: 

• a basic,  though  limited,  understanding  of  what  a centrally  planned  economy — as  in  the 
Soviet  Union  before  1990 — is,  of  the  meaning  of  quality  of  life,  of  primary  industry  in  an 
economy,  and  of  aspects  of  one’s  lifestyle  (see  question  38) 

• an  ability  to  make  simple  inferences. 

Many  students  achieving  at  this  standard  had  difficulty 

• recalling  and  applying  some  basic  knowledge  as  outlined  in  the  course  of  study  (see 
question  31) 

• analysing,  synthesizing,  and  evaluating  sources  of  a more  abstract  nature. 


Standard  of  Excellence — Sample  Questions  and  Commentary 


Use  the  information  below  to  answer  questions  7 and  9. 


Source  I 

Nearly  every  immigrant  wanted  to  work  and  this  caused  tension  between 
Amen  can-bom  workers  and  newcomers.  Part  of  this  conflict  resulted  from  a 
lack  of  understanding  of  the  causes  of  unemployment.  Many  American-born 
workers  did  not  understand  that  they  were  being  displaced  because  the 
tremendous  changes  in  manufacturing  had  caused  the  mechanization  of  their 
jobs. 


Source  II 

Amen  can-bom  workers  resented  the  immigrants,  who  were  willing  to  work  for 
less  money.  They  grew  angry  because  they  felt  their  demands  for  higher  wages 
and  better  working  conditions  were  threatened  by  immigrants  who  were  eager  to 
work  under  any  conditions.  The  competition  between  American-bom  workers 
and  immigrants  became  intense  as  immigration  increased  and  the  worker 
population  expanded  one- third  faster  than  the  total  population. 


7.  The  MAIN  point  in  source  I is  that 

16.9  A.  immigration  led  to  the  mechanization  of  American  industries 
4.0  B.  workers  tended  to  be  more  close-minded  and  racist  than  other  people 

65.4  *C.  workers  lost  their  jobs  to  machines  but  blamed  immigration 

13.6  D.  owners  hired  more  immigrants  because  they  were  willing  to  work  for 
less 

Question  7 required  students  to  identify  the  main  point  in  source  I.  A fair 
percentage  of  students  achieving  at  the  acceptable  standard  were  able  to  identify 
the  main  idea,  whereas  a high  percentage  of  students  achieving  at  the  standard  of 
excellence  were  able  to  do  so. 


9.  Based  on  both  sources,  what  generalization  can  be  made  about  immigra- 
tion and  economic  growth? 

35.1  A.  People  move  to  areas  where  the  economy  is  expanding. 

53.4  *B.  Workers  often  believe  that  immigration  creates  unemployment. 

5.0  C.  More  skilled  workers  than  unskilled  workers  tend  to  immigrate. 

6.3  D.  Immigration  has  lowered  profits  and  slowed  the  growth  of  factories. 

Question  9 required  students  to  use  a higher  level  cognitive  process.  They 
had  to  select  the  generalization  that  could  be  made  based  on  the  two  sources. 
Many  students  achieving  at  the  acceptable  standard  and  some  students 
achieving  at  the  standard  of  excellence  had  problems  selecting  the  correct 
generalization.  Too  many  students  in  both  groups  were  drawn  to  the  first 
alternative. 
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Students  achieving  at  the  standard  of  excellence  could 

• readily  extract  information  from  a reading  passage  and  use  it  to  support  their  main 
idea 

• identify  an  author’s  point  of  view  in  a reading  passage  (see  question  7) 

• make  judgments  using  their  knowledge  of  economic  growth 

• show  a clear  understanding  of  the  effects  of  technology;  for  example,  computer  use  in 
industry 

• show  a good  understanding  of  the  economic,  social,  and  environmental  implications  of 
technology  on  our  way  of  life. 

Students  achieving  at  this  standard  had  difficulty 

• interpreting  source  documents  that  presented  highly  abstract  ideas  and  synthesizing 
the  information  (see  question  9) 

• recalling  some  basic  facts  and  concepts. 


Written  Response 


Assignment 

Recycling  is  a growing  concern  in  Alberta. 

HOW  SHOULD  THE  GOVERNMENT  OF 
ALBERTA  BE  INVOLVED  IN  RECYCLING 
PROGRAMS? 

In  presenting  your  opinion,  consider  economic  and 
environmental  factors. 

Your  written  response  may  take  the  form  of  a letter 
to  the  government,  a speech  to  your  class,  a newspaper 
editorial,  or  a position  paper. 

To  support  your  opinion,  use  examples  and  evidence 
from  the  Information  to  Consider  on  page  25,  as  well 
as  from  YOUR  KNOWLEDGE  OF  ECONOMIC 
SYSTEMS. 


Although  the  quality  of  writing  in  social  studies  is 
better  in  1991  than  in  1987,  not  enough  students  are 
meeting  the  acceptable  standard.  Teachers  were  disap- 
pointed with  many  of  the  papers  that  they  marked.  The 
markers’  specific  observations  include  the  following: 

•The  majority  of  students  had  difficulty  focusing  on  the 
issue.  Writing  was  general  rather  than  specific. 

• A number  of  students  confused  the  levels  of  govern- 
ment. 

• Most  students  could  state  a position  but  had  difficulty 
communicating  ideas  in  support  of  their  position. 


Grade  9 Social  Studies 
Percentage  Distribution  of  Scores 
by  Reporting  Category  for  Written  Response 


SCORE  SCALE 
POINTS 

REPORTING  CATEGORY 

Support  of  Position 

Expression 

5 

(Excellent) 

3.8 

3.7 

4 

(Good) 

15.9 

19.1 

3 

(Acceptable) 

2 

42.6 

52.6 

(Limited) 

1 

(Poor) 

INS 

29.0 

203 

7.8 

3.4 

(Insufficient  or  No 

Response) 

1.0 

1.0 

*the  shaded  portion  indicates  below-standard 
performance 


• Students  wrote  with  conviction;  their  writing  indicated 
a sincere  concern  for  the  environment.  They  believed 
that  government  should  play  an  important  role  in 
solving  recycling  problems  but  that  government  is  not 
fulfilling  this  function  adequately. 

• A small  number  of  clearly  excellent  papers  were  writ- 
ten. The  quality  of  thought  and  expression  in  these 
papers  far  exceeded  the  expectations  for  Grade  9 Social 
Studies. 

The  Student  Evaluation  Branch  is  preparing  a booklet 
comprising  samples  of  the  full  range  of  student  writing, 
the  standards  used  in  marking,  and  a discussion  of  the 
relative  strengths  and  weaknesses  in  student  writing. 
Booklets  will  be  sent  to  schools  that  have  students  en- 
rolled in  Grade  9. 
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SECTION  5 

ACHIEVEMENT  BY  GENDER 


Information  on  the  gender  of  students  who 
wrote  the  provincial  achievement  tests  has 
been  collected  and  data  have  been  analysed 
and  reported  since  1989. 

For  the  June  1991  administration,  results  are 
presented  in  Figures  5-1  to  5-3. 

This  section  of  the  report  answers  the  follow- 
ing questions: 

• What  is  the  proportion  of  males  and 
females  who  wrote  Grade  3 Science, 


Grade  6 Mathematics,  and  Grade  9 Social 
Studies  achievement  tests  in  June  1991? 

• Is  the  percentage  of  males  and  females 
meeting  standards  the  same  in  each 
subject? 

• What  is  the  pattern  of  male  and  female 
achievement  across  grades  3,  6,  and  9? 

• Are  the  1991  results  for  individual  sub- 
jects similar  to  or  different  from  those  of 
1989  and  1990? 


Figure  5-1 


NUMBER  OF  ACHIEVEMENT  TESTS  WRITTEN  BY  GENDER* 

June  1991 


(thousands) 


Male 


Female 


* Figures  do  not  include  tests  where  students  did  not  identify  their  gender.  The  number  of  students  who  did  not 
provide  gender  data  is  as  follows: 

Grade  3 Science — 16  students 
Grade  6 Mathematics — 7 students 
Grade  9 Social  Studies — 58  students 
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Figure  5-2 


PERCENTAGE  OF  STUDENTS  ACHIEVING  ACCEPTABLE  STANDARDS 
OR  HIGHER  ON  BOTH  COMPONENTS  OF  THE  TESTS  BY  GENDER 

June  1991 


111  Male  ^ Female 


*80%  of  students  were  expected  to  achieve  the  acceptable  standards  or  higher  on 
both  components  of  the  test. 


Figure  5-3 


PERCENTAGE  OF  STUDENTS  ACHIEVING  STANDARDS  OF 
EXCELLENCE  ON  BOTH  COMPONENTS  OF  THE  TESTS  BY  GENDER 


j~~j  Male  ^ Female 


*10%  of  students  were  expected  to  achieve  the  standards  of  excellence  on  both 
components  of  the  test. 
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OBSERVATIONS  AND  DISCUSSION 


Number  of  Students  Writing  the  Tests 


Results  for  1991  reveal  that  more  boys  than 
girls  wrote  the  achievement  tests  at  each 
grade  level.  The  pattern  for  grades  3 and  6 is 
supported  by  the  1989  and  the  1990  data.  For 
the  Grade  9 data,  the  proportion  of  boys  and 
girls  changed  over  time. 

In  1989,  the  number  of  boys  was  marginally 
higher  than  the  number  of  girls.  Then  in 


1990,  the  proportions  reversed.  This  change 
and  an  explanation  of  the  change  in  Grade  9 
requires  further  investigation  to  include 
absentee  rates,  enrolments,  and  age  by  gen- 
der. This  information  may  be  used  to  identify 
factors  that  may  have  contributed  to  the 
change. 


Percentage  of  Students  Achieving  Standards 


Results  for  1991  reveal  that  levels  of  achieve- 
ment are  virtually  the  same  for  boys  and  girls 
for  Grade  3 Science  as  well  as  for  Grade  9 
Social  Studies.  For  Grade  6 Mathematics, 
boys  outperformed  girls,  particularly  for  the 
standard  of  excellence.  The  pattern  for  the 
Grade  6 Mathematics  is  supported  by  the 
1990  Grade  3 Mathematics  results. 

The  pattern  for  the  1991  Grade  3 Science 
results  is  not  supported  by  the  1990  Grade  6 
Science  results  or  the  1989  Grade  9 Science 
results,  where  boys  outperformed  girls,  par- 
ticularly for  the  standard  of  excellence,  in 
both  years. 


This  change  in  pattern  of  achievement  for 
boys  and  girls  also  exists  in  social  studies, 
where  there  is  virtually  no  difference  in  levels 
of  achievement  for  boys  and  girls  in  the  1991 
Grade  9 Social  Studies  results  but  Grade  6 
girls  outperformed  boys  in  1989. 

An  explanation  of  these  changes  requires 
further  investigation  that  will  include  more 
subjects  and  more  administrations  of  tests  of 
the  same  subjects.  This  will  be  possible  when 
further  data  become  available  in  1992. 
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SECTION  6 

ACHIEVEMENT  BY  AGE 


Information  on  the  age  of  students  who  wrote 
the  provincial  achievement  tests  was  first 
collected  in  June  1990.  The  1990  data  were 
analysed  and  reported  in  the  1990  Achieve- 
ment Testing  Program  Provincial  Report. 
Similar  information  was  collected  for  the  June 
1991  adminstration.  Results  are  presented  in 
Table  6-1,  and  Figures  6-1  and  6-2. 

This  section  of  the  report  answers  the  follow- 
ing questions: 


• What  is  the  age  distribution  of  students 
who  wrote  the  1991  Achievement  Tests? 

• What  relationship,  if  any,  does  age  appear 
to  have  with  achievement  as  measured  by 
the  1991  provincial  achievement  tests? 

• What  changes  in  age  distribution  and 
achievement  are  evident  between  the  1991 
and  the  1990  test  results? 


Table  6-1 

Number  of  Achievement  Tests  Written  by  Age 
June  1991 


Age* 

Grade  3 Science 

Grade  6 Mathematics 

Grade  9 
Social  Studies 

(years) 

Number 

Percentage 

Number 

Percentage 

Number 

Percentage 

Under  7 

4 

0.0 









7 

46 

0.1 

— 

— 

— 

— 

8 

15  121 

44.7 

— 

— 

— 

— 

9 

16  705 

49.4 

3 

0.0 

— 

— 

10 

1780 

5.3 

54 

0.2 

— 

— 

11 

89 

0.3 

14  349 

44.9 

— 

— 

12 

12 

0.0 

15  248 

47.8 

1 

0.0 

13 

3 

0.0 

2 082 

6.5 

91 

0.3 

14 

2 

0.0 

149 

0.5 

13  007 

47.0 

15 

— 

— 

15 

0.0 

12  599 

45.6 

16 

— 

— 

4 

0.0 

1727 

6.2 

17 

— 

— 

— 

— 

91 

0.3 

i is 

— 

— 

— 

— 

11 

0.0 

Over  18 
No  Age 

— 

— 

— 

— 

5 

0.0 

Information 

75 

0.2 

26 

0.1 

114 

0.4 

Total 

33  837 

100.0 

31  930 

100.0 

27  646 

100.0 

*Age  on  the  test  day  (June  11, 1991). 

The  dashes  in  each  column  indicate  that  there  were  no  students. 
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Figure  6-1 
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Figure  6-2 


ACHIEVEMENT  BY  AGE* ** 

PERCENTAGE  OF  STUDENTS  ACHIEVING  STANDARDS 
OF  EXCELLENCE  ON  BOTH  COMPONENTS  OF  THE  TESTS 

June  1991 


Grade  3 Science 


Grade  6 Mathematics 


Grade  9 Social  Studies 


*Results  for  groups  of  fewer  than  25  students  are  not  shown  on  these  graphs. 

**10%  of  students  were  expected  to  achieve  the  standards  of  excellence  on  both  components  of  the  test. 
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OBSERVATIONS  AND  DISCUSSION 


Age  Groups 


The  1991  data  reveal  that  within  each  grade 
level,  slightly  over  90  per  cent  of  students 
split  into  two  age  groups  of  approximately 
equal  size: 


Grade  6: 11-year  olds  and  12-year-olds 
Grade  9: 14-year  olds  and  15-year-olds 

The  1990  data  exhibited  the  same  patterns. 


Grade  3:  8-year-olds  and  9-year-olds 


Student  Achievement 


The  1991  data  reveal  that  there  is  a high 
negative  relationship  between  age  and  stu- 
dent achievement.  This  means  that  within  a 
grade  level,  younger  students’  achievement  is 
higher  than  older  students’  achievement. 

This  is  true  for  both  the  acceptable  standards 
and  the  standards  of  excellence  across  all 
three  grades-.  The  same  patterns  and  rela- 
tionships appeared  in  the  1990  results  as  well. 
This  lower  achievement  of  older  students  in 
the  grade  levels  tested  may  be  attributed  to 
the  interaction  of  several  factors.  The  most 
likely  factors  are  speculated  to  be  repeating 
students,  late  entrants  into  formal  schooling, 
and  transferees  from  other  provinces  or 
countries  where  there  may  be  curricular 
differences  and/or  the  language  of  instruction 
is  not  their  first  language. 


Schools  and  jurisdictions  whose  results  reflect 
negative  relationships  similar  to  the  ones 
exhibited  by  the  provincial  results  are  encour- 
aged to  identify  the  specific  factors  operating 
within  their  own  classrooms  and  schools. 
Accurate  identification  of  such  factors  would 
be  the  first  step  in  the  development  of  plans  to 
reduce  or  eliminate  this  unwelcome  relation- 
ship. 

Student  Evaluation  Branch  welcomes  obser- 
vations from  school  administrators  on  the 
factors  they  identify  as  being  the  major  con- 
tributors to  the  prevailing  negative  relation- 
ships between  age  and  achievement. 
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SECTION  7 

ACHIEVEMENT  BY  GRADE  LEVEL 


Section  7 is  an  exploratory  approach  to  re- 
porting results  from  the  1991  achievement 
tests. 

This  section  attempts  to  provide  answers  to 
the  following  question: 

What  percentage  of  grades  3,  6,  and  9 
Alberta  students 

•may  be  achieving  beyond  grade  level 
•are  achieving  at  grade  level 
•may  be  achieving  at  grade  level 
•are  not  yet  achieving  at  grade  level 

as  measured  by  the  June  1991  achieve- 
ment tests? 

To  facilitate  implementation  of  a results- 
based  curriculum,  an  attempt  is  made  in  this 
section  to  analyse  and  report  student  achieve- 
ment by  "levels". 

Within  each  curriculum,  specific  outcomes, 
expressed  as  the  knowledge  and  skills  to  be 
acquired  and  developed,  are  arranged  into 
sequences  reflecting  the  developmental  na- 
ture of  learning  and  expectations  for  a learn- 
er’s increasing  competence  or  confidence  in 
the  subject  area. 

As  students  differ  from  each  other  in  their 
rates  of  learning  or  development  along  the 
continuum,  the  levels  of  learning  can  be 
considered  to  be  independent  of  age  or  the 
amount  of  time  spent  in  school. 

Additionally,  since  all  students  do  not  neces- 
sarily achieve  in  chronological  sequences  but 
learn  in  different  ways,  the  flexibility  of  a 
levels  approach  allows  teachers  to  focus  at  all 
times  on  the  individual  needs  of  students. 

In  summary,  a results-based  curriculum  is 
vitally  important  to  the  educational  process 
as  it  allows  for  and  encourages  progression 
while  also  accommodating  individual  differ- 
ences. 


The  1991  achievement  tests  were  designed  to 
measure  student  achievement  within  the 
given  grade  level.  However,  there  were  a 
number  of  questions  on  the  achievement  tests 
that  could  be  considered  to  measure  more 
advanced  levels  of  performance.  For  reporting 
purposes  only,  we  have  classified  students 
within  the  levels  framework,  based  on  the 
following  four  definitions: 

1.  May  Be  Achieving  Beyond  Grade  Level 

Students  who  may  be  achieving  beyond 
grade  level  are  those  students  who  achieved 
the  standards  of  excellence  for  both  major 
components  of  the  test  and  correctly  an- 
swered certain  selected  questions  that 
assess  more  advanced  levels  of  performance. 

2.  Achieving  At  Grade  Level 

Students  who  are  achieving  at  grade  level 
are  those  students  who  achieved  the  accept- 
able standards  for  both  major  components 
of  the  test  and  those  students  who  achieved 
the  standards  of  excellence  but  did  not 
answer  correctly  the  selected  questions  that 
assess  more  advanced  levels  of  performance. 

3.  May  Be  Achieving  At  Grade  Level 

Students  who  may  be  achieving  at  grade 
level  are  those  students  who  met  the  accept- 
able standard  for  one  of  the  major  compo- 
nents of  the  test  but  not  for  both  compo- 
nents. 

4.  Not  Yet  Achieving  At  Grade  Level 

Students  who  are  not  yet  achieving  at  grade 
level  are  those  students  who  did  not  meet 
the  acceptable  standard  for  either  major 
component  of  the  test.  These  students  are 
not  yet  considered  to  be  performing  at  grade 
level. 
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Tables  7-1,  7-2,  and  7-3  present  results  from  this  new  analysis  for  grades  3,  6,  and  9 respec- 

tlvely'  Table  7-1 

Grade  3 Science  Achievement  Test 
Distribution  by  Grade  Level  Achievement 
June  1991 


Level 

Number 

Per  Cent 

Students  Who  May  Be  Achieving  Beyond  Grade  3 Level 

1742 

5.1 

Students  Who  Are  Achieving  At  Grade  3 Level 

21  970 

65.0 

Students  Who  May  Be  Achieving  At  Grade  3 Level 

5 362 

15.8 

Students  Who  Are  Not  Yet  Achieving  At  Grade  3 Level 

4 763 

14.1 

Total 

33  837 

100.0 

Table  7-2 

Grade  6 Mathematics  Achievement  Test 
Distribution  by  Grade  Level  Achievement 
June  1991 


Level 

Number 

Per  Cent 

Students  Who  May  Be  Achieving  Beyond  Grade  6 Level 

2 319 

7.3 

Students  Who  Are  Achieving  At  Grade  6 Level 

19  685 

61.6 

Students  Who  May  Be  Achieving  At  Grade  6 Level 

4 537 

14.2 

Students  Who  Are  Not  Yet  Achieving  At  Grade  6 Level 

5 389 

16.9 

Total 

31  930 

100.0 

Table  7-3 

Grade  9 Social  Studies  Achievement  Test 
Distribution  by  Grade  Level  Achievement 
June  1991  


Level 

Number 

Per  Cent 

Students  Who  May  Be  Achieving  Beyond  Grade  9 Level 

807 

2.9 

Students  Who  Are  Achieving  At  Grade  9 Level 

17  967 

65.0 

Students  Who  May  Be  Achieving  At  Grade  9 Level 

4 974 

18.0 

Students  Who  Are  Not  Yet  Achieving  At  Grade  9 Level 

3 898 

14.1 

Total 

27  646 

100.0 

These  results  quantify  the  wide  range  of  student  achievement  within  a nominal  grade  level. 
The  importance  of  this  experimental  analysis  is  that,  for  the  first  time,  it  shows  performance 
relationship  to  provincial  expectations  based  on  a “levels”  curricula  approach. 
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SECTION  8 

ACHIEVEMENT  OVER  TIME 


An  important  goal  of  Alberta  Education  is  to 
measure  and  report  changes  in  student 
achievement  over  time.  Since  1983,  the 
Student  Evaluation  Branch  has  been  measur- 
ing the  achievement  of  students  in  grades  3,  6, 
and  9 in  the  subject  areas  of  language  arts, 
mathematics,  science,  and  social  studies.  The 
administration  of  the  Achievement  Testing 
Program  follows  a four-year  cycle  for  each 
subject  within  each  grade  level.  As  a result, 
in  1983,  1987,  and  1991,  science  was  tested  in 
Grade  3,  mathematics  was  tested  in  Grade  6, 
and  social  studies  was  tested  in  Grade  9.  This 
section  of  the  report  answers  the  question 


• Has  achievement  as  measured  by  individual 
provincial  achievement  tests  in  Grade  3 
Science,  Grade  6 Mathematics,  and  Grade  9 
Social  Studies  changed  since  1983  and  1987? 

Two  studies  were  carried  out.  The  first  study 
was  designed  to  compare  achievement  since 
1983  on  the  multiple-choice  questions  in 
Grade  3 Science,  Grade  6 Mathematics,  and 
Grade  9 Social  Studies.  The  second  study  was 
designed  to  compare  achievement  since  1987 
on  the  written-response  part  of  the  Grade  9 
Social  Studies  test.  The  results  of  these 
special  studies  follow. 


ACHIEVEMENT  OVER  TIME:  MULTIPLE  CHOICE 
Study  Design 


In  the  past,  achievement-over-time  studies 
carried  out  by  the  Student  Evaluation  Branch 
involved  the  re-administration  of  earlier  tests 
to  a sample  of  students  who  were  also  writing 
the  current  tests.  Because  these  students  had 
to  write  two  tests,  the  re-administration  was 
either  one  week  before  or  one  week  after  the 
current  test  administration.  However,  moti- 
vational levels  and  state  of  preparation  were 
likely  different  from  when  the  tests  were 
written  during  the  original  administrations. 
As  well,  the  additional  testing  time  required 
of  students  participating  in  the  study  was  a 
concern. 

To  address  these  issues,  a new  design  was 
implemented  for  the  1991  achievement-over- 
time study.  Random  samples  of  students 
wrote  all  or  part  of  previous  achievement  tests 
under  the  same  conditions  as  the  earlier  test 
administrations.  That  is,  the  achievement- 
over- time  tests  were  administered  to  the 
random  samples  at  the  same  time  students  in 
the  rest  of  the  province  were  writing  the 


current  achievement  test.  To  maintain  the 
validity  of  the  tests  for  the  1991  students,  the 
achievement-over-time  tests  were  developed 
to  match  the  blueprints  of  the  1991  tests  by 
selecting  questions  from  the  previous  tests 
that  were  still  appropriate  in  1991.  Questions 
that  were  no  longer  appropriate,  because  of 
curricular  changes,  were  replaced  with  1991 
questions.  As  a result,  it  was  possible  to  use 
the  1983  and  1987  Grade  3 Science  tests  in 
their  entirety,  but  only  parts  of  the  Grade  6 
Mathematics  and  Grade  9 Social  Studies  tests 
were  still  appropriate.  As  well,  a subset  of 
students  who  were  randomly  selected  to 
participate  in  the  achievement-over- time 
study  wrote  the  current  1991  tests  so  that  we 
could  verify  that  the  samples  in  the  study 
were  representative  of  the  province. 

Table  8-1  shows  the  composition  of  the 
achievement-over-time  ( AOT ) tests  in  terms  of 
the  number  of  questions  they  contain  from  the 
1983,  1987,  and  1991  tests. 
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Table  8-1 

Comparison  of  the  Achievement-Over-Time  Tests 
by  the  Number  of  Questions  from  the  Previous 
and  Current  Tests 


1983 

AOT  Test 

1987 

AOT  Test 

1991 

AOT  Test 

Grade  3 Science 

Number  of  1983  Questions 

50 

0 

0 

Number  of  1987  Questions 

0 

50 

0 

Number  of  1991  Questions 

0 

0 

50 

Grade  6 Mathematics 

Number  of  1983  Questions 

32 

0 

0 

Number  of  1987  Questions 

0 

40 

0 

Number  of  1991  Questions 

23 

15 

55 

Grade  9 Social  Studies 

Number  of  1983  Questions 

22 

0 

0 

Number  of  1987  Questions 

0 

20 

0 

Number  of  1991  Questions 

38 

39* 

60 

*One  question  on  the  Grade  9 Social  Studies  1987  AOT  test  was  dropped  because  of  a typographical 


error. 


Methodology 


Sampling  of  the  schools  involved  in  the  study 
was  carried  out  independently  in  grades  3,  6, 
and  9.  Because  of  project  design  considera- 
tions, only  schools  with  24  or  more  students  in 
regular  English  programs*  who  wrote 
achievement  tests  in  1990  in  the  grade  being 
sampled  were  included  in  the  groups  from 
which  samples  were  drawn.  At  each  of  the 
three  grade  levels,  all  eligible  schools  were 
identified  and  listed.  From  each  of  these  lists, 
random  samples  of  schools  were  selected.  The 
sample  sizes  were  targeted  for  approximately 
3 000  students  at  each  grade  level.  Since  the 
sampling  was  carried  out  independently  at 
each  grade  level,  some  schools  were  randomly 
selected  to  participate  at  more  than  one 
grade. 


Before  test  materials  were  shipped,  the  par- 
ticipating schools  submitted  to  the  Student 
Evaluation  Branch  copies  of  their  class  lists. 
Within  each  class,  students  were  randomly 
assigned  to  write  either  the  AOT  1983  test, 
the  AOT  1987  test,  or  the  AOT  1991  test. 
Along  with  test  materials,  the  class  lists  were 
sent  back  to  the  schools  with  instructions 
about  which  test  each  student  was  to  write. 

The  students  who  wrote  the  1983  achieve- 
ment-over-time test  are  referred  to  hereafter 
as  the  AOT83  group,  the  students  who  wrote 
the  1987  achievement-over- time  test  are 
referred  to  as  the  AOT8  7 group,  and  the 
students  who  wrote  the  1991  test  are  called 
the  AOT91  group  to  differentiate  them  from 
the  students  in  the  rest  of  the  province  who 
wrote  the  1991  regular  achievement  test. 


*See  the  first  footnote  on  page  9. 
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Results 


Before  comparisons  were  made  between  the 
AOT83  group  and  the  students  writing  in 
1983,  and  the  AOT87  group  and  the  students 
writing  in  1987,  it  was  necessary  to  verify 
that  the  students  selected  to  participate  in  the 
special  study  were  representative  of  students 
throughout  the  province.  Since  approximately 
one-third  of  the  students  sampled  were 


assigned  to  the  AOT91  group,  it  was  possible 
to  compare  their  results  with  the  provincial 
results.  Because  the  large  sample  sizes 
allowed  us  to  detect  very  small  differences, 
only  differences  where  the  probability  was 
less  than  0.01  were  considered  to  be  signifi- 
cant. Table  8-2  presents  the  findings. 


Table  8-2 

Comparison  of  Results  for  the  AOT91  Groups 
with  Provincial  Results 


AOT91 

Results 

Provincial 

Results 

Grade  3 Science 

Raw  Score  Mean 

33.8 

34.4 

Standard  Deviation 

9.1 

8.9 

Number  of  Students 

1 169 

29  610 

Grade  6 Mathematics 

Raw  Score  Mean 

37.1 

37.9 

Standard  Deviation 

10.4 

10.4 

Number  of  Students 

1 119 

26  107 

Grade  9 Social  Studies 

Raw  Score  Mean 

37.7 

37.6 

Standard  Deviation 

11.0 

10.3 

Number  of  Students 

989 

21  971 

Table  8-2  shows  that  at  all  three  grades,  the 
means  of  the  AOT91  group  and  of  the  province 
were  within  one  raw  score  point.  None  of 
these  differences  was  statistically  significant 
at  the  0.01  level.  As  there  were  no  significant 
differences  between  the  means  of  the  AOT91 
groups  and  the  provincial  means,  we  con- 
cluded that  the  samples  of  students  in  the 
special  study  were  representative  of  all  regu- 
lar students  in  the  province. 


Because  Grade  6 Mathematics  and  Grade  9 
Social  Studies  students  in  the  special  study 
responded  to  a number  of  1991  questions,  it 
was  possible  to  carry  out  an  additional  check 
that  the  random  assignment  resulted  in 
groups  that  were  representative  of  the  prov- 
ince. Table  8-3  presents  the  results  of  these 
comparisons. 
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Table  8-3 

Comparison  of  AOT  Samples  with  Provincial  Results 


AOT83 

Group 

Province 

AOT87 

Group 

Province 

Grade  6 Mathematics 

No.  of  1991  Questions 

23 

23 

15 

15 

Raw  Score  Mean 

14.9 

15.0 

9.6 

9.4 

Standard  Deviation 

4.8 

4.7 

3.3 

3.3 

Number  of  Students 

1083 

26  107 

1 079 

26  107 

Grade  9 Social  Studies 

No.  of  1991  Questions 

38 

38 

39 

39 

Raw  Score  Mean 

23.8 

23.9 

24.0 

24.1 

Standard  Deviation 

6.8 

6.6 

7.4 

7.1 

Number  of  Students 

964 

21  971 

973 

21  971 

On  the  1991  questions,  the  differences  be- 
tween the  means  of  the  students  in  the  special 
study  and  the  means  of  students  in  the  rest  of 
the  province  are  within  0.2  raw  score  points, 
as  can  be  seen  from  Table  8-3.  These  differ- 
ences are  not  statistically  significant  at  the 
0.01  level.  Based  on  these  analyses,  we 
concluded  that  the  random  assignment  was 
successful  and  that  the  samples  are  repre- 
sentative of  the  province. 

Since  it  was  established  that  our  samples 
were  representative  of  the  province,  we  were 
able  to  proceed  with  the  comparisons  between 
the  1991  students  in  the  study  and  the  stu- 
dents in  1983  and  1987.  As  stated  earlier,  in 
the  methodology  section,  we  restricted  our 
sampling  to  schools  with  an  estimated  mini- 
mum size  of  24  students  in  the  grade  being 
sampled.  When  making  comparisons  with  the 
previous  years,  we  attempted  to  exclude 


students  who  wrote  in  the  earlier  years  who 
were  from  schools  that  were  smaller  than  the 
smallest  school  size  in  the  1991  samples. 
However,  individual  student  data  were  not 
available  for  the  1983  Grade  3 Science  and 
Grade  6 Mathematics  tests.  It  was  not  possi- 
ble, therefore,  to  exclude  from  these  compari- 
sons students  from  schools  smaller  than  the 
school  sizes  in  the  1991  sample. 

In  the  comparisons,  data  from  all  1983  Grade 
3 Science  and  Grade  6 Mathematics  tests 
were  used. 

Table  8-4  shows,  for  each  grade,  the  number 
of  common  questions  that  were  answered  by 
both  the  1991  students  and  the  students  in 
the  previous  years,  the  means  and  standard 
deviations  for  these  questions,  and  the 
number  of  students  on  which  these  statistics 
were  based. 
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Table  8-4 

Achievement  Over  Time 
Comparison  of  Descriptive  Statistics 


AOT83 
Group 
(wrote  in 
1991) 

Regular 

1983 

Students 

AOT87 
Group 
(wrote  in 
1991) 

Regular 

1987 

Students 

Grade  3 Science 

Number  of  Questions 

50 

50 

50 

50 

Raw  Score  Mean 

42.7* 

39.2 

39.7* 

35.7 

Standard  Deviation 

5.4 

N/A** 

8.5 

7.9 

Number  of  Students 

1 119 

23  902 

1 122 

31  087 

Grade  6 Mathematics 

Number  of  Questions 

32 

32 

40 

40 

Raw  Score  Mean 

21.6* 

19.3 

29.9* 

28.6 

Standard  Deviation 

6.4 

N/A** 

6.9 

6.8 

Number  of  Students 

1 083 

25  621 

1 079 

28  573 

Grade  9 Social  Studies 

Number  of  Questions 

22 

22 

20 

20 

Raw  Score  Mean 

14.3* 

13.8 

13.3 

13.4 

Standard  Deviation 

4.2 

4.2 

4.1 

3.9 

Number  of  Students 

964 

19  299 

973 

24  671 

*The  mean  for  the  1991  students  in  the  AOT  group  is  significantly  different  from  the  mean  of  the  previous  admin- 
istration at  the  0.001  level. 

**The  original  student  response  data  for  1983  were  not  available.  Question  difficulties  were  used  to  reconstruct  the 
means,  but  standard  deviations  could  not  be  reconstructed. 


Table  8-4  shows  that  for  all  but  one  com- 
parison of  interest,  the  1991  students  scored 
significantly  higher  than  the  students  in 
1983  and  1987.  The  exception  is  Grade  9 
Social  Studies,  where  the  achievement  in 
1991  is  the  same  as  the  achievement  in 
1987.  Specifically,  Grade  3 Science  students 
in  1991  achieved  between  3.1  and  3.9  raw 
score  points  higher  than  the  1983  popula- 
tion based  upon  a 99%  confidence  interval. 
This  means  that  the  chances  are  99  in  100 
that  the  difference  between  all  1991  stu- 
dents in  regular  English  Grade  3 Science 
programs  and  the  1983  students  falls  within 
the  range  specified.  Similarily,  Grade  3 
Science  students  in  1991  achieved  between 


3.3  and  4.6  raw  score  points  higher  than  the 
1987  population.  Grade  6 Mathematics  stu- 
dents achieved  between  1.8  and  2.8  raw  score 
points  higher  in  1991  than  the  1983  popula- 
tion, and  between  0.8  and  1.8  raw  score  points 
higher  in  1991  than  the  1987  population. 
Grade  9 Social  Studies  students  achieved 
between  0.2  and  0.9  raw  score  points  higher  in 
1991  than  the  1983  population,  but  their 
achievement  was  not  significantly  different 
from  the  1987  population. 

More  detailed  analyses  of  the  changes  in 
achievement  are  described  below  for  each 
grade. 
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Grade  3 Science 


A comparison  of  the  results  of  the  1983  test 
written  in  1983  and  again  in  1991  indicates 
that  more  students  in  1991  answered  ques- 
tions correctly  than  students  did  in  1983. 
Although  higher  percentages  of  students 
achieved  greater  success  in  both  process  skills 
and  subject  matter,  the  strength  of  the  1991 
Grade  3 Science  students  was  unquestionably 
in  the  area  of  process  skills. 

In  1991,  the  increase  in  the  percentage  of 
students  correctly  answering  1983  process 
skills  questions  was  dramatic: 

• Approximately  30%  more  students  could 
accurately  collect  data  using  their  obser- 
vation skills.  They  also  knew  the  differ- 
ence between  observations  and  infer- 
ences. 

• Approximately  15%  more  students  could 
successfully  group  unfamiliar  objects 
according  to  physical  characteristics, 
using  their  knowledge  and  understand- 
ing of  classification. 

• Approximately  10%  more  students  could 
recognize  correct  graphical  representa- 
tions of  data  and  infer  correct  interpreta- 
tions from  the  graphed  data. 

• Approximately  13%  more  students  knew 
how  to  measure  the  length  and  volume  of 
objects  accurately. 

Students  in  1991  also  showed  marked  im- 
provement over  1983  students  in  their  knowl- 
edge and  understanding  of  science  concepts: 

• Approximately  25%  more  students 
understood  the  capacity  of  objects  made 
of  different  materials  to  float  or  sink  in 
water. 

• Approximately  18%  more  students 
understood  that  some  reactions  can  be 
reversed  and  others  cannot. 

• Approximately  26%  more  students  knew 
the  difference  between  materials  that 


naturally  decompose  and  those  that  do  not. 

A comparison  of  the  results  of  the  1987  test 
written  in  1987  and  again  in  1991  indicates 
that  student  achievement  has  improved.  An 
examination  of  the  question-by-question 
results  revealed  that  although  achievement 
remained  the  same  on  some  questions,  on 
others  a much  larger  percentage  of  1991 
students  answered  correctly  compared  with 
1987  students.  Students’  ability  in  the  area 
of  process  skills  has  improved  since  1987: 

• Approximately  15%  more  students  could 
use  their  senses  to  accurately  collect  data 
based  on  observation. 

• Approximately  10%  more  students  could 
measure  volume  accurately,  rank  objects 
based  on  comparative  measurements  of 
mass,  and  read  temperatures  on  a ther- 
mometer. 

• Approximately  10%  more  students  had 
the  ability  to  make  interpretations  from 
graphs  and  infer  beyond  the  data  pro- 
vided. 

• Approximately  15%  more  students  could 
classify  objects  into  groups  based  on 
physical  characteristics. 

Although  the  greatest  improvement  was  in 
the  area  of  process  skills,  students  also  im- 
proved in  the  area  of  subject  matter.  Approxi- 
mately 10%  more  students  had  an  under- 
standing of: 

• the  differences  between  living  and 
nonliving  things. 

• shadows  and  how  they  are  affected  by 
the  apparent  shift  of  the  sun’s  position 
relative  to  the  Earth’s  rotation. 

• how  to  save  energy  in  the  home. 

• what  the  process  of  dissolving  means. 
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One  possible  reason  for  the  dramatic  improve- 
ment in  achievement  compared  with  1983  and 
1987  is  that  teachers  are  doing  a better  job  of 
delivering  science  programs  across  Alberta, 
particularly  in  the  teaching  of  process  skills. 
Another  reason  is  that  teachers  and  students 
have  greater  access  to  resources  and  support 
that  are  imperative  for  an  effective  science 


Grade  6 Mathematics 

A greater  increase  in  student  achievement 
occurred  from  1983  to  1991  than  from  1987  to 
1991  in  Grade  6 Mathematics.  Specifically, 
increases  over  1983  were  evident  in  the  fol- 
lowing strands  and  objectives: 

• Numeration — understanding  of  numbers, 
fractions,  and  percentages. 

• Measurement — recognition  of  time  on  the 
24-hour  clock,  knowledge  of  metric 
lengths,  and  application  of  perimeter  and 
area. 

• Geometry — knowledge  of  terminology 
and  recognition  of  a segment  and  a slide. 

• Graphing — application  of  knowledge  to 
solve  questions  using  a bar  graph. 

By  strand,  most  increases  in  achievement 
were  in  Numeration  and  Measurement.  No 
important  changes  were  found  among  ques- 
tions measuring  the  strands  Operations  and 
Properties,  and  Problem  Solving. 

Under  the  cognitive  level  taxonomy,  there  was 
an  overall  increase  betweeen  1983  and  1991. 
Large  increases  were  noted  on  single  ques- 
tions in  the  areas  of  Comprehension  and 
Problem  Solving,  and  large  increases  were 
noted  on  several  questions  in  the  areas  of 
Knowledge  and  Application. 


program.  Still  another  reason  is  the  current 
overlap  of  process  skills  in  science  with  other 
subject  areas,  such  as  social  studies  and 
mathematics.  This  could  account  for  the 
greater  increase  in  process  skills  compared 
with  subject  matter. 


Interestingly,  there  was  no  achievement 
increase  in  routine  procedural  questions. 
Questions  requiring  quick  routine  calculations 
were  answered  correctly  by  roughly  the  same 
or  a slightly  smaller  proportion  of  students  in 
1991  as  in  1983.  This  was  evident  in  the  non- 
calculator section. 

Increases  in  achievement  since  1987  were 
generally  not  large.  Large  increases  were 
evident  in  only  three  questions  from  the 
following  strands: 

• Measurement — approximately  10%  more 
students  were  able  to  solve  a problem 
related  to  finding  the  volume  of  a box. 

• Graphing — approximately  10%  more 
students  were  able  to  recognize  similar 
shapes  in  a slide  image. 

• Problem  Solving — approximately  15% 
more  students  were  able  to  solve  a prob- 
lem related  to  the  understanding  of 
number  patterns. 

Similarly,  the  achievement  increase  from 
1987  to  1991  with  regard  to  cognitive  levels 
was  small  but  general.  The  three  questions 
referred  to  above  were  in  the  areas  of  Knowl- 
edge (1)  and  Problem  Solving  (2). 
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In  summary,  for  Grade  6 Mathematics,  our 
analysis  seems  to  indicate  that  there  was  a 
general  but  small  improvement  in  all  strands, 
with  the  least  improvement  in  the  area  of 
Operations  and  Properties.  More  emphasis 
seems  to  have  been  placed  by  teachers  and 
administrators  on  understanding  and  the 
application  of  skills  and  knowledge,  and  less 
emphasis  on  routine  operations  without 
applications.  Some  causes  for  the  greater 
increase  over  1983  than  over  1987  may  be 
that  the  current  curriculum  was  introduced 
shortly  before  the  1983  test,  whereas  it  had 
been  in  operation  for  five  years  by  the  time  of 
the  1987  tests.  Teachers  and  education 
leaders,  and  therefore  students,  were 


Grade  9 Social  Studies 

A slight  overall  increase  in  student  achieve- 
ment was  found  from  1983  to  1991.  An  ex- 
amination of  the  responses  to  the  individual 
1983  questions  revealed  that  only  three 
questions  showed  large  increases  (10%  or 
greater)  in  the  percentage  of  students  answer- 
ing correctly  compared  with  1983.  These  three 
questions  were  based  on  the  knowledge  objec- 
tives. 

The  reason  for  the  achievement  increase  in 
two  of  the  questions  could  be  that  students  in 
1991  have  a stronger  vocabulary  than  stu- 
dents did  in  1983.  The  key  words  in  these 
questions  were  “scarcity”  and  “bias,”  and 
students  needed  to  know  the  meaning  of  these 
terms  to  answer  the  questions  correctly.  The 
other  question  that  showed  improvement 
dealt  with  the  Soviet  Union.  The 


increasingly  aware  of  what  was  expected. 

It  is  interesting  that  in  the  basic  facts  portion 
of  the  test,  there  was  a noticeably  consistent, 
albeit  small,  decline  from  1987  to  1991  (1983 
is  not  comparable).  In  addition,  subtraction, 
multiplication,  division,  and  mixed  operations, 
the  average  number  of  questions  attempted 
and  the  average  number  answered  correctly 
declined  slightly  in  1991.  We  note  that  the 
questions  were  not  identical  for  1987  and 
1991  but  were  of  similar  type  and  expected 
difficulty.  The  results  may  be  related  to  the 
reduced  emphasis  on  routine  operations 
without  applications. 


achievement  increase  might,  in  part,  have 
resulted  from  the  intense  media  coverage  of 
the  events  of  the  past  year  in  the  Soviet 
Union.  Interestingly,  one  question  showed  a 
decrease  in  achievement  of  approximately 
14%  between  1983  and  1991.  Two  of  the 
response  options  to  this  question  contained 
the  word  “materialism.”  It  is  possible  that 
with  the  shift  in  the  focus  of  the  curriculum 
from  “materialism”  to  “quality  of  life,”  1991 
students  found  this  question  to  be  more 
difficult  because  they  were  not  familiar  with 
the  word. 

No  large  differences  in  student  responses 
were  found  between  1987  and  1991. 


Conclusions 


Ideally,  when  comparing  achievement  over 
time  the  same  test  is  given  over  different  time 
periods  to  samples  of  students  who  have  not 
gained  access  to  the  test  ahead  of  time.  Since 
achievement  tests  are  released  sifter  adminis- 
tration, it  is  not  possible  to  guarantee  that 
students  writing  in  subsequent  administra- 
tions have  not  seen  the  test.  When  developing 
the  new  study  design,  consideration  was 


given  to  asking  teachers  whose  students  were 
participating  in  the  study  not  to  familiarize 
students  with  the  previous  achievement  tests. 
However,  we  decided  that  would  not  be  fair  to 
the  students  involved,  as  it  can  be  useful  for 
students  to  know  the  types  of  questions  they 
will  be  asked  to  answer.  We  realized  at  the 
time  that  this  was  a limitation  with  the  design. 
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A telephone  survey  of  a sample  of  teachers 
whose  classes  participated  in  the  study  re- 
vealed that  the  overwhelming  majority  of 
teachers  did  not  have  the  1983  tests  to  prac- 
tice with  because  the  tests  did  not  remain  in 
schools  after  the  1983  administration.  The 
few  teachers  who  stated  that  they  did  have  a 
copy  of  the  1983  test  indicated  that  the  1983 
test  was  not  given  much  attention.  However, 
almost  one-half  of  the  teachers  surveyed 
indicated  that  they  had  gone  through  all  or 
part  of  the  1987  test  with  their  students  to 


provide  them  with  an  understanding  of  the 
format  of  the  test.  Very  few  teachers  indi- 
cated that  they  used  the  1987  achievement 
test  as  a practice  test.  Since  for  the  grade  6 
and  9 achievement-over-time  tests  selected 
items  from  the  previous  tests  were  embedded 
with  1991  items,  any  effect  of  practice  should 
have  been  minimized.  Regardless,  the  1987 
comparisons,  especially  for  Grade  3 Science, 
should  be  interpreted  with  caution.  We  are 
confident  that  the  1983  comparisons  are  valid. 


ACHIEVEMENT  OVER  TIME:  WRITTEN  RESPONSE 
Study  Design 


Students’  writing  performance  has  been  of 
particular  interest  to  educators  and  to  the 
public  since  the  beginning  of  the  Achievement 
Testing  Program.  As  a consequence  of  inter- 
est in  how  well  students  write,  a study  was 
initiated  to  compare  1987  writing  with  1991 
writing  in  Grade  9 Social  Studies.  This  was  a 
qualitative  or  descriptive  study  that  required 
the  teacher-readers  to  take  a research  or 
"reader-as-observer"  stance  toward  the  papers 
they  read,  rather  than  the  usual  evaluative  or 
"reader-as-assessor"  stance  of  the  teacher- 
marker.  No  attempt  was  made  to  rescore 
papers;  rather,  teacher- readers  described 
features  of  1987  and  1991  writing  in  two 
general  categories: 

Defence  of  Position 
— development  of  ideas 
— persuasiveness 
— accuracy  of  content 


Use  of  Language 
— vocabulary  choices 
— control  of  conventions:  spelling, 
punctuation,  grammar 
— organization  and  structure 

In  1987,  the  written-response  assignment 
involved  short  answers  and  one  extended 
response.  In  1991,  the  assignment  was  lim- 
ited to  one  extended  response.  In  order  to 
facilitate  comparison  of  similar  assignments, 
only  the  extended  response  from  1987  was 
compared  with  the  1991  assignment. 

Comparisons  were  made  at  two  standards: 
Acceptable  ( 3 ),  which  represents  work  at  an 
acceptable  level  for  students  completing 
Grade  9,  and  Excellent  (5),  which  represents 
outstanding  work  for  students  completing 
Grade  9. 
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Methodology 


Papers  read  in  the  study  were  selected  at 
random  from  June  1987  papers  that  received 
scores  of  Acceptable  (3)  or  Excellent  (5)  and 
from  June  1991  papers  that  received  scores  of 
Acceptable  (3)  or  Excellent  (5). 

A group  of  ten  experienced  teacher-readers, 
representing  all  major  regions  of  the  province 
and  a variety  of  school  settings,  reviewed  the 
selected  papers. 


Working  alone  and  in  pairs,  and  then  discuss- 
ing papers  as  a group,  the  teacher-readers 
described  the  papers  for  features  of  defence  of 
position  and  use  of  language. 

The  teacher-readers  then  compared  their 
descriptions  of  1991  and  1987  Acceptable  (3) 
and  Excellent  (5)  papers  and  drew  conclusions. 


Results 


The  results  of  the  study  are  given  in  tables 
8-5  and  8-6.  Each  comment  describes  the 
features  of  what  students  writing  at  a par- 
ticular level  did  in  1991  compared  with 
students  in  1987.  The  wording  in  the  tables 
comes  directly  from  teacher-reader’s  descrip- 
tions. 

The  level  of  expectations  embedded  in  the 
1991  scoring  criteria  was  increased  signifi- 
cantly from  the  1987  criteria.  However,  the 


work  described  at  each  level  met  the  standard 
established  for  the  appropriate  marking 
session.  Teacher-readers  agreed  that  papers 
judged  to  be  Acceptable  (3)  were  considerably 
better  in  1991  than  in  1987.  As  well,  a higher 
percentage  of  students  were  able  to  achieve 
the  Acceptable  (3)  level  in  1991  than  in  1987. 
Improvements  were  noted  as  well  in  the 
papers  scored  Excellent  (5)\  however,  fewer 
students  achieved  this  level  in  1991  than  in 
1987. 
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Table  8-5 

Acceptable  (3)  Papers  in  Social  Studies  9 


KEY  FEATURES  OF  1991 

KEY  FEATURES  OF  1987 

ACCEPTABLE  (3)  PAPERS 

ACCEPTABLE  (3)  PAPERS 

Defence  of  Position 

Defence  of  Position 

• could  identify  with  issue  on  a personal 
level,  focused 

• did  not  relate  to  topic,  sense  of  remoteness 
from  topic 

• genuine  concern,  commitment  toward  topic 

• lack  of  understanding  of  issue 

• use  of  own  experiences  to  support  state 
ments 

• limited  defence  of  position,  support  was 
superficial 

• use  of  prior/media/other  subject  knowledge 
common 

• did  not  incorporate  information  from  outside 
social  studies  classroom 

• tendency  to  "broaden"  topic  to  include  other 
environmental  concerns 

• identified  with  one  of  the  speakers,  not  with 
a position 

• insight  shown  but  not  consistent 

• lacking  in  insight 

• sense  of  urgency  regarding  topic 

• emotionalism  rather  than  conviction 

• simplification  of  concepts  common 

• basic  social  studies  concepts  generally 
understood  but  in  a concrete  manner  only 

• ideas  generally  organized 

• ideas  poorly  organized 

• confusion  over  government  levels  and  roles 

Note:  the  question  did  not  require  students  to 
address  levels  and  roles  of  government 

Use  of  Language 

Use  of  Language 

• paragraphing  skills  evident 

• poor  organization  and  structure 

• specific  words,  dynamic  vocabulary  at- 
tempted 

• limited  vocabulary 

• general  attempt  to  use  and  spell  social 
studies  terms  correctly 

• frequent  errors  in  spelling,  punctuation,  and 
grammar 

• environmental  "buzz  words"  and  jargon 
common 

• "Motherhood"  statements  common 
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Table  8-6 

Excellent  (5)  Papers  in  Social  Studies  9 


KEY  FEATURES  OF  1991 

KEY  FEATURES  OF  1987 

EXCELLENT  (5)  PAPERS 

EXCELLENT  (5)  PAPERS 

Defence  of  Position 

Defence  of  Position 

• encouraged  personal  involvement/identified 
with  topic 

• attitude  of  detachment  from  topic 

• insightful,  creative  thought  exhibited 

• little  personal  insight 

• focused,  controlled,  does  not  wander 

• no  creativity  in  approach  to  topic  or  possible 
solutions 

• social  studies  concepts  incorporated 

• limited  inclusion  of  social  studies  concepts 

• convincing,  persuasive  arguments 

• mechanical  argumentation 

• qualitative  evaluation,  used  data  well 

• position  supported  with  multiple  arguments 

• more  than  one  idea  presented 

• examples  very  specific 

• clear  delineation  between  roles  of  govern- 
ment/business/public 

• good  use  of  data  from  outside  social  studies 
classroom 

• genuine  concern  with  topic,  healthy 
skepticism  exhibited 

• both  sides  of  question  understood 

• occasional  meaningless  regurgitation  of 
statistics  to  support  argument 

• acknowledgment  of  flaws  in  argument  or  of 
the  opposite  view,  and  rebuttal 

• understood  government  levels,  roles,  limita- 
tions 

• good  understanding  of  economic  systems 

Note:  the  question  did  not  require  students  to 
address  government  levels,  roles,  limitations 
nor  did  it  make  reference  to  economic  systems 

Use  of  Language 

Use  of  Language 

• experimental  use  of  vocabulary 

• advanced  technical  vocabulary 

• sense  of  audience,  with  careful  attention  to 
spelling  and  conventions 

• good  spelling  and  grammar 

• variety  in  forms  of  expression 

• logical  organization  and  good  structure 

• dramatic  language  used  but  generally  well 
controlled 

• accurate  but  pedantic  language  used 
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Conclusions 


The  question  that  guided  the  study — Has 
writing  by  Grade  9 Social  Studies  stu- 
dents improved  from  1987  to  1991? — was 

answered  in  the  affirmative.  The  1991  stu- 
dents at  the  Acceptable  (3)  level  of  perform- 
ance produced  better  writing  than  did  their 
1987  counterparts.  Although  a lower  percent- 
age of  students  were  able  to  achieve  the 
Excellent  (5)  level  of  performance  in  1991, 
those  who  did  so  produced  better  writing. 

A change  in  the  scoring  criteria  between  1987 
and  1991  reflected  an  increase  in  standards 
and  in  the  level  of  expectations  for  Grade  9 
Social  Studies  students.  This  may  have 
contributed  to  the  lower  percentage  of  stu- 
dents able  to  achieve  a score  of  Excellent  (5). 
However,  even  with  this  increase  in  standards 
and  criteria,  a significantly  greater  number  of 
students  were  able  to  achieve  a score  of  Ac- 
ceptable (3).  One  possible  conclusion  is  that 
while  top-range  students’  writing  has  not 
changed  markedly  since  1987,  average-range 
students’  writing  is  better. 

A greater  variety  of  response  formats  may 
also  have  contributed  to  the  improved  writing 
on  the  1991  test.  Students  in  1991  could 
choose  to  write  a letter,  speech,  editorial,  or 
position  paper,  whereas  1987  students  were 
limited  to  a prescribed  letter  format. 

In  1991,  students  addressed  only  one  written 
response,  whereas  in  1987  students  answered 


four  short  written-response  questions  before 
attempting  the  extended  response.  This  may 
have  contributed  to  the  higher  number  of  INS 
(insufficient)  responses  in  1987.  Within  the 
confines  of  a test  situation,  some  students 
may  have  experienced  oversaturation  with  the 
topic  before  attempting  the  extended  re- 
sponse. 

Teachers  participating  in  the  study  theorized 
that  the  greatest  factor  in  improved  writing 
was  related  to  the  topic  itself.  The  teachers 
felt  that  the  1987  topic — To  What  Extent 
Should  the  Natural  Resources  of  the  Eastern 
Slopes  be  Developed? — had  little  sense  of 
immediacy  or  relevancy  for  many  students. 
The  commitment  to  a topic  that  lends  itself  to 
more  involved  argumentation  and  more 
persuasive  writing  was  missing. 

In  contrast,  familiarity  with  the  1991  topic — 
How  Should  the  Government  of  Alberta  be 
Involved  in  Recycling  Programs? — allowed 
students  to  tie  their  experiences  and  convic- 
tions to  the  writing  assignment.  Such  a 
personal  connection  to  the  issue  may  partially 
account  for  the  greater  percentage  of  students 
able  to  achieve  an  Acceptable  (3)  score.  That 
this  broader  1991  topic  was  also  a more  acces- 
sible topic  is  evidenced  by  the  greater  percent- 
age of  students  responding  to  it.  In  1987, 

3.8%  of  students  produced  INS  papers, 
whereas  in  1991  only  1%  did:  almost  all  1991 
students  felt  competent  to  address  this  topic. 
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A remarkably  high  percentage  of  students 
achieved  an  Acceptable  (3)  or  higher  score  for 
Use  of  Language  on  the  1991  test.  Teachers 
concluded  that  this  might  be  attributed  to  two 
factors: 

•a  greater  emphasis  on  student  writing  in  all 
subjects  and  at  all  grades  levels 
•social  studies  classroom  instruction  and 
practice  in  effectively  communicating  ideas 
central  to  the  development  of  issues 

Given  that  assignments  have  become 
cognitively  more  challenging,  yet  much  more 


open-ended,  and  that  standards  have  become 
more  demanding  and  still  students’  work  has 
improved,  one  might  conclude  that  classroom 
instruction  that  fosters  critical  thinking  and 
clear,  concrete  expression  of  ideas  has  been 
the  greatest  factor  in  improving  student 
writing. 

Detailed  discussion  of  the  study  design, 
methodology,  results,  and  conclusions  can  be 
obtained  by  calling  the  Achievement  Testing 
Unit  of  Student  Evaluation  Branch,  Alberta 
Education,  at  427-0010. 
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SECTION  9 

DIRECTIONS:  BROADENED  ASSESSMENT 


PURPOSE 


Broadened  assessment  is  a strategy  that 
includes  the  use  of  different  evaluation  meth- 
ods and  instruments  designed  to  collect  a 
broader  range  of  information  about  what 
students  know  and  can  do.  Along  with  paper 
and  pencil  tests,  broadened  assessment  will 
assist  us  in  creating  a more  complete  picture 
of  what  students  learn  and  how  they  apply 
those  learnings.  This  strategy  is  designed  to 
assess  students  in  a manner  that  reflects  the 
way  students  learn  and  to  support  effective 
classroom  teaching. 


During  the  1990-91  school  year,  the  Achieve- 
ment Testing  Unit  of  the  Student  Evaluation 
Branch  initiated  the  development  of  many 
broadened  assessment  strategies.  With  the 
assistance  of  classroom  teachers,  the  assess- 
ment instruments  were  validated  for  field 
testing  purposes  in  the  spring  of  1991.  The 
number  of  students  who  participated  in  each 
field  test  ranged  from  150  to  1 350.  A brief 
description  of  each  instrument  and  some 
general  findings  based  on  student  results 
follows. 


GRADE  3:  ACROSS-CURRICULUM  PORTFOLIO  ASSESSMENT 

The  portfolio  project  was  designed  to  answer  the  question,  At  what  level  are  Grade  3 
students  performing  in  oral  communication,  problem-solving,  participation,  and 
process  skills? 

Oral  Communication  Skills 


This  component  involved  the  audio  taping 
and/or  video  taping  of  an  oral  presentation  by 
Grade  3 students  in  classrooms  across  Al- 
berta. Students  could  make  their  presenta- 
tion individually  or  in  small  groups  to  the 
students  in  their  classrooms.  Presentations 
could  range  from  puppetry  or  storytelling  to 
interviewing  or  reporting  but  did  not  have  to 
be  language  arts  material  specifically. 

Through  an  analysis  of  the  audio  and  video 
tapes,  we  learned  that  students  working  at 
level  show  a move  toward  independence; 
however,  occasional  prompting  and/or  support 
is  needed.  These  students  have  a beginning 
awareness  of  audience,  purpose,  the  need  to 
project  their  voices,  and  the  importance  of 
voice,  face,  and  body  language.  The  tasks 
they  undertake  are  basic.  Their  props,  when 
used,  are  simple,  and  they  may  have  some 
difficulty  using  them.  They  are  starting  to 
show  more  confidence  in  their  ability;  hence, 
they  are  only  somewhat  nervous  in  front  of  an 
audience. 


Students  working  beyond  level  demonstrate 
self-control  and  can  work  independently. 
These  students  need  minimal  support  and  are 
willing  to  take  risks.  They  display  confidence 
in  their  ability  and  have  developed  awareness 
of  audience,  purpose,  and  logical  sequence. 
They  recognize  the  importance  of  voice,  face, 
and  body  language. 

Students  not  yet  at  level  exhibited  few  of  the 
characteristics  of  the  students  at  level.  They 
were  reluctant  participants,  usually  very 
nervous  and  intimidated  by  an  audience. 

The  teachers  involved  in  this  evaluation 
thought  that  the  assessment  of  oral  communi- 
cation skills  in  language  learning  has,  for  the 
most  part,  been  neglected.  They  also  thought 
that  the  basic  premise  of  the  study  was  a good 
one  and  suggested  several  modifications  that, 
when  incorporated,  should  result  in  an 
assessment  instrument  of  improved  quality. 
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Problem-Solving  Skills 


This  component  was  designed  to  gather  infor- 
mation about  Grade  3 students’  problem- 
solving skills  and  strategies.  Teachers  were 
asked  to  select,  with  their  students,  a prob- 
lem-solving activity  and  use  Polya’s  problem- 
solving model.  The  activity  required  the 
students  to  use  insight,  lent  itself  to  a variety 
of  solutions,  and  involved  a mathematical 
process  or  skill. 

Teachers  developed  a holistic  scoring  guide  to 
evaluate  students’  work. 

We  learned  that  most  students  performing  at 
level  could: 

• identify  the  question  being  asked 

• identify  relevant  and  key  words 


• develop  a plan  to  find  an  answer 

• use  the  plan  to  find  an  answer 

Students  performing  beyond  level  could  do 
all  of  the  above  as  well  as: 

• restate  the  problem  in  their  own 
words 

• write  a number  sentence  to  match  the 
problem 

• recognize  that  there  is  more  than  one 
way  to  solve  a problem 

• identify  and  solve  multiple-step 
problems 

Students  not  yet  at  level  required  constant 
prompting  and  support  to  engage  in  and 
complete  the  tasks. 


Participation  Skills 


Classroom  teachers  assessed  their  students’ 
participation  skills  through  the  use  of  the  skill 
development  charts  found  in  the  Social  Stud- 
ies Program  of  Studies.  Teachers  agreed  that 
the  checklist  approach  would  be  the  most 
effective  and  manageable  way  to  collect  the 
information. 

After  the  checklists  were  reviewed  and  the 
information  was  compiled,  teachers  agreed 
that  students  participating  at  level. 

• understand  the  need  for  rules  and/or 
for  observing  them 

• often  show  respect  for  the  rights  and 
opinions  of  others 

• may  be  easily  distracted  but  attention 
can  be  focused  back 

• understand  the  difference  between  a 
leader  and  a follower 

Students  participating  beyond  level  had  all 
of  the  above  characteristics  and: 

• are  actively  involved  in  making  group 
rules 

• always  show  respect  for  the  rights 
and  opinions  of  others 


• stay  focused  despite  distractions  from 
others 

• accept  a leadership  role  as  necessary 

Those  students  not  yet  at  level: 

• generally  believe  that  rules  are  for 
others 

• are  unable  to  stay  on  task 

• find  it  difficult  to  take  turns  and 
participate  cooperatively 

• have  difficulty  understanding  the 
difference  between  leader  and  fol- 
lower 

Teachers  strongly  support  this  method  of 
assessing  student  participation  skills  because 
it  supports  the  teaching  and  assessment  of  an 
important  part  of  the  curriculum  that  has 
been  neglected.  Teachers  also  suggested 
minor  revisions  to  the  checklist  and  the  level 
descriptors  in  an  effort  to  improve  the  quality 
and  reliability  in  the  information  that  this 
instrument  gathers. 
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Process  Skills 


This  study  required  students  to  use  materials 
provided  in  a baggie  to  complete  seven  tasks 
that  were  designed  to  assess  their  ability  to 
perform  process  skills  outlined  in  Division  I 
Science.  Analytic  scoring  criteria  were  devel- 
oped, validated,  and  applied  by  teachers. 
Results  indicate  that  students  performing  at 
level  could: 

• measure  using  arbitrary  units  and 
measure  accurately 

• select  appropriate  devices  for  collect- 
ing data 

• predict  from  graphed  data 

• classify  natural  objects  according  to 
only  one  property 

Student  performing  beyond  level  could  also: 

• organize  data  in  a graph 

• rank  objects  from  heaviest  to  lightest 

• classify  natural  objects  using  more 
than  one  property 

• provide  written  descriptions  of  meth- 
ods of  classification 


Students  that  were  not  yet  at  level  could: 

• rank  most  objects  from  heaviest  to 
lightest 

• classify  some  natural  objects  using 
only  one  property 

Teachers  felt  that  this  method  of  assessment 
was  extremely  useful  and  informative  in  that 
it: 

• supported  classroom  teaching 

• provided  a meaningful  context  for  learn- 
ing and  assessment 

• provided  results  that  could  be  used  to 
improve  instruction 

In  addition,  they  said  that  students  didn’t  feel 
as  though  they  were  being  tested.  Instead, 
students  commented  that  they  were  learning 
something  as  they  progressed  through  each 
activity. 


GRADE  3:  ACROSS-CURRICULUM  THEMATIC  TEST 


This  field  test  focused  on  a theme  integrating 
Grade  3 science,  mathematics,  social  studies 
and  language  arts.  New  questions  reflecting  a 
context  that  students  can  easily  relate  to  were 
included.  The  theme  was  chosen  to  reflect  the 
way  many  students  learn  and  teachers  teach 
in  the  classroom.  An  eight-and-one-half-inch 
by  fourteen-inch  information  sheet  about  the 
seasons  provided  context  for  the  test  and 
information  for  the  student  that  could  be  used 
to  answer  each  question. 

The  questions  were  of  different  designs, 
ranging  from  shading  in  graphs  to  indicate 
the  number  of  visitors  to  the  park  in  winter  to 
shading  in  the  number  of  coins  that  would  buy 
a toy  boat  worth  90  cents.  For  the  writing 
assignment,  students  were  able  to  make  a 
choice.  They  could  either  write  a report,  using 
the  facts  provided,  about  the  beaver  and  why 
beavers  were  good  swimmers,  or  write  a story 
about  their  favorite  season,  telling  why  it  is 
their  favorite. 


The  following  are  comments  made  by  Grade  3 
teachers: 

“Reflects  integration  of  subject  areas — 
theme-based  just  like  lessons.” 

“I  think  this  is  a great  idea — it  really  fits  in 
with  the  way  primary  education  is  done — 
thematically!” 

“This  test  is  fun  and  interesting  to  do.  I 
think  children  would  enjoy  this  test  and 
would  find  it  challenging.” 

Only  one  teacher  believed  that  this  type  of 
test  was  inappropriate.  Students  who  wrote 
the  test  commented  that  it  was  interesting, 
hard,  and  made  them  think.  Field  testing  this 
instrument  resulted  in  valuable  input  from 
teachers  and  students  that,  when  incorpo- 
rated into  the  test,  will  provide  an  assessment 
instrument  of  better  quality. 
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GRADE  6:  MATHEMATICS  AND  SCIENCE  PERFORMANCE -BASED  ASSESSMENT 


This  assessment  was  designed  to  answer  the 
question,  Can  performance  assessment 
broaden  the  description  of  what  stu- 
dents know  and  can  do  in  the  applica- 
tion of  their  problem-solving  and/or 
science  process  skills?  Performance-based 
assessment  engages  students  in  the  use  of 
"hands-on"  materials  to  complete  meaningful 
real-life  activities.  Although  there  was  no 
time  limit,  students  were  encouraged  to 
complete  each  activity  within  10  to  15  min- 
utes. 

Scoring  criteria  were  used  to  describe  what 
students  demonstrated  they  knew  and  could 
do.  Most  students  were  able  to: 

• infer  possible  effects  of  change  within 
an  ecosystem. 

• infer  the  path  that  electricity  travels 
in  hidden  circuits 

• construct  simple  electrical  circuits 
using  a bulb,  wire,  and  a dry  cell 

• classify  animals  as  predator  or  prey 
on  the  basis  of  their  special  adapta- 
tions 

• make  predictions  and  inferences 
based  on  direct  observation  and 
recorded  data 

• locate  the  outline  of  a two-dimen- 
sional picture  using  ordered  pairs 

• use  manipulatives  to  determine  the 
shortest  route  between  two  points  on 
a map 


• rank  objects  from  heaviest  to  lightest 
using  an  equal-arm  balance 

• convert  tally  information  to  equiva- 
lent fractions  and  decimals 

• construct  bar  graphs  to  accurately 
reflect  data  collected 

• create  classifications  for  data  collected 

• find  and  compare  volume  of  rectangu- 
lar prisms 

Classroom  teachers  felt  that  this  type  of 
assessment  provides  accurate  information 
about  what  students  can  actually  do  and  how 
well  they  can  do  it. 

The  following  are  comments  made  by  stu- 
dents: 

“It  was  easy,  big,  and  interesting.” 

“I  thought  that  this  was  great,  especially  the 
science  part  where  you  had  to  join  wires  and 
make  light  bulbs  work.” 

“In  math  I liked  the  K that  you  had  to  flip, 
slide,  and  turn.” 

“It’s  a lot  better  than  multiple-choice  tests 
because  you  have  things  to  work  with  in- 
stead of  a paper  and  pencil.” 

After  the  teacher  validation  comments  are 
reviewed  further,  some  activities  may  have  to 
be  revised  and  perhaps  new  ones  developed 
for  field  testing  in  spring  1992. 


GRADE  6:  VIEWING  AND  LISTENING  SKILLS  ASSESSMENT 

This  assessment  was  designed  to  answer  the  question,  At  what  level  are  students  in 
Grade  6 performing  in  listening  and  viewing  skills? 


Listening  Skills 


The  listening  skills  component  required 
students  to  listen  to  three  pieces  of  literature. 
After  hearing  the  fiction  and  nonfiction  selec- 
tions, students  answered  some  multiple- 


choice  questions.  After  hearing  the  poetry 
piece,  they  retold  what  they  heard.  In  addi- 
tion, students  listened  to  some  directions  and 
did  what  the  directions  asked  them  to  do. 
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The  information  collected  was  analysed  by 
Grade  6 teachers.  They  also  used  information 
provided  by  teachers  who  field  tested  the 
instrument  to  describe  student  expectations 
for  "at  level"  and  "not  yet  at  level"  perform- 
ances. 

Students  who  are  at  level  are  able  to  retell  a 
great  many  of  the  significant  details  and 
events.  Most  of  the  time,  the  sequencing  of 
the  events  and/or  details  is  correct.  These 
students  understand  the  denotative  meanings 
of  words  they  have  heard;  however,  they  may 
experience  more  difficulty  with  the  connota- 
tive  meaning  of  words.  At  level  students  can 
paraphrase  a spoken  message  but  will  experi- 
ence some  difficulty  doing  so.  They  can  also 
follow  oral  directions  but  will  make  a few 
errors  in  the  process.  The  at  level  students 
generally  lack  some  of  the  confidence  in  their 
general  listening  skills  typically  found  in  the 
beyond  level  students.  As  a result,  their 
critical  listening  skills  are  not  quite  so  well 
developed. 


Students  who  are  beyond  level  are  consist- 
ently able  to  retell  all  significant  details  and 
events  in  proper  order.  These  students 
understand  both  denotative  and  connotative 
meanings  of  words  that  they  have  heard. 
Beyond  level  students  can  paraphrase  a 
spoken  message  with  no  difficulty  and  follow 
oral  directions  with  no  errors.  In  addition, 
they  display  confidence  and  competence  in 
their  general  listening  skills.  As  a result, 
they  are  excellent  critical  listeners  as  well. 

Students  who  are  not  yet  at  level  are  usually 
able  to  retell  only  a few  of  the  significant 
details  and  events.  Students  functioning  at 
this  level  have  a confused  idea  of  sequence. 
These  students  typically  have  a difficult  time 
understanding  all  but  the  most  basic  vocabu- 
lary. Connotative  meanings  are  not  under- 
stood at  all.  Paraphrasing  for  these  students 
is  difficult  because  they  probably  had  consid- 
erable difficulty  understanding  what  they 
heard. 


Viewing  Skills 


For  the  viewing  skills  component  of  the  as- 
sessment, students  were  given  two  pictures. 
They  had  to  describe  what  they  saw  in  each 
picture  and  then  compare  and  contrast  how 
they  felt  when  they  looked  at  both  pictures. 

In  addition,  they  had  to  look  at  a political 
cartoon  and  explain  what  message  they  got 
from  it. 

The  information  collected  was  analysed  by 
Grade  6 teachers.  From  the  analysis,  teachers 
were  able  to  describe  student  performances 
that  reflect  curriculum  expectations  for  at 
level , beyond  level , and  not  yet  at  level 
performances. 


Those  students  who  are  functioning  at  level 
are  able  to  communicate  in  writing  informa- 
tion they  derive  from  a visual  presentation; 
however,  their  statements  tend  to  be  general. 
The  general  knowledge  base  of  these  students 
is  not  as  broad  as  those  performing  beyond 
level.  When  asked  to  describe  a picture,  these 
students  are  able  to  pick  out  general  details, 
but  the  subtleties  will  escape  their  detection. 
When  students  performing  at  level  examine 
pictures  that  display  different  emotions,  they 
have  some  difficulty  interpreting  the  various 
moods  and  feelings  portrayed.  The  at  level 
students  can  compare  and  contrast  different 
pictures  only  if  the  pictures  are  relatively 
straightforward  and  not  too  detailed. 


-75- 


Students  who  are  functioning  beyond  level 
are  able  to  communicate  in  writing  informa- 
tion they  derive  from  a visual  presentation. 
The  information  is  generally  quite  sophisti- 
cated because  these  student  bring  a broad 
general  knowledge  base  to  their  understand- 
ing. When  asked  to  describe  a picture,  stu- 
dents performing  beyond  level  are  able  to 
pick  out  the  most  minute  details.  They  are 
also  able  to  examine  pictures  displaying 
different  emotions  and  interpret  the  various 
moods  and  feelings  portrayed.  These  beyond 
level  students  show  an  ability  to  compare  and 
contrast  different  pictures,  and  provide  many 
appropriate  details. 

Students  not  yet  at  level  demonstrate  a 
limited  understanding  of  what  they  view. 


Thus,  they  are  able  to  communicate  in  writing 
in  only  a superficial  manner.  When  asked  to 
describe  a picture,  these  students  pick  out 
only  the  most  obvious  features  of  the  picture. 
Students  not  yet  at  level  have  a great  deal  of 
difficulty  interpreting  moods  and  feelings 
portrayed  in  pictures  displaying  various 
emotions.  These  students  are  not  able  to 
compare  and  contrast  different  pictures  unless 
the  pictures  are  very  basic. 

This  type  of  assessment  was  received  very 
positively.  In  the  spring  of  1992,  student 
viewing  and  listening  skills  will  be  assessed 
on  a sample  basis  and  reported  next  year  with 
the  Grade  6 English  Language  Arts  Achieve- 
ment Test  results. 


GRADE  9:  MATHEMATICS  AND  SCIENCE  PERFORMANCE -BASED  ASSESSMENT 


This  assessment  was  designed  to  answer  the 
question,  Can  performance  assessment 
broaden  the  description  of  what  students 
know  and  can  do  in  the  application  of 
their  problem-solving  and/or  science 
process  skills?  Although  there  was  no  time 
limit,  students  were  encouraged  to  complete 
each  activity  within  10  to  15  minutes. 

Scoring  criteria  were  used  to  describe  what 
students  knew  and  could  do.  Students  were 
able  to: 

• find  the  square  root  of  a number  using 
estimation 

•understand  and  use  a strategy  for  finding 
the  surface  area  of  any  right  prism 
•interpret  maps  and  scale  drawings 
•use  ratios  to  solve  problems 
•use  a dichotomous  key  to  classify  animals 

• determine  experimentally  which  of  two 
materials  is  a better  insulation,  which  wall 
design  conserves  heat  energy  best,  and 
which  of  two  liquids  is  best  to  use  in  a cool- 
ing system 


•construct  series  and  parallel  electrical  cir- 
cuits 

• determine  the  effect  of  temperature  on  rates 
of  reaction 

The  performance  of  many  Grade  9 students 
was  disappointing.  They  experienced  diffi- 
culty in  arriving  at  solutions  that  were  cor- 
rect. This  may  be  the  result  of  a weakness  in 
lab  skills  and/or  use  of  manipulative. 

However,  Grade  9 students  enjoyed  the  as- 
sessment and  commented  that  it  was  better  to 
have  their  hands  actually  on  the  materials  to 
solve  problems  or  explain  concepts  rather 
than  imagine  everything  from  a diagram  on 
paper.  Teachers  saw  ways  in  which  these 
activities  could  be  easily  incorporated  into 
classroom  instruction.  In  addition,  they  saw 
how  important  aspects  of  the  curriculum  could 
be  emphasized  when  incorporated  into  this 
type  of  assessment. 
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GRADE  9:  PARTICIPATION  SKILLS  ASSESSMENT 


This  assessment  was  designed  to  answer  the 
question,  At  what  level  are  students  in 
Grade  9 performing  in  participation 
skills?  Ten  teachers  from  across  Alberta 
used  three  different  forms  or  checklists  to 
indicate  the  intrapersonal  skills,  interper- 
sonal relations,  and  group  process  and  discus- 
sion skills  for  each  student.  Accompanying 
the  forms  was  a five-point  rating  scale,  where 
1 indicated  no  evidence  that  the  student 
possessed  the  skill  and  5 represented  student 
independence  and  control. 

The  ratings  assigned  to  students  by  teachers 
participating  in  the  assessment  provided  a lot 
of  detail  about  the  participation  skills  of 
students  in  Grade  9.  From  this  information, 
descriptions  of  at  level,  beyond  level,  and  not 
yet  at  level  performance  of  students’  partici- 
pation skills  have  been  developed. 

Students  performing  at  level  deal  well  with 
minor  frustrations  and  seek  assistance  when 
needed.  These  students  often  make  good 
choices  and  have  the  ability  to  empathize  with 
others.  However,  students  performing  at 
level  tend  to  feel  that  they  have  the  best 
opinion  and  sometimes  are  impulsive  in 
responding.  These  students  compromise 
occasionally  and  accept  disagreement.  In 
groups,  they  co-operate  actively  with  others 
and  work  as  directed  by  the  teacher  and/or 
group  leader.  Occasionally,  students  perform- 
ing at  level  will  monopolize  discussions. 

Students  performing  beyond  level  deal  with 
personal  frustration  in  a positive  manner, 
seek  help  on  more  difficult  assignments,  and 
make  logical  choices.  They  also  have  an 
internal  drive  to  challenge  themselves.  These 
students  see  how  behavior  affects  others. 

They  can  be  given  a task  and  left  to  work  at  it 


independently.  In  addition,  beyond  level 
students  respond  positively  to  the  opinions  of 
others  and  defend  the  rights  of  others.  They 
wait  to  be  recognized  before  speaking,  spend 
time  helping  others  freely,  and  seek  compro- 
mise. Students  performing  beyond  level 
display  leadership  skills  in  group  discussions. 
They  are  always  on  task  and  lead  others  by 
example. 

Students  performing  not  yet  at  level  express 
anger  and  lack  self-control  when  frustrated. 
They  often  refuse  assistance  and  refuse  to 
make  decisions.  These  students  criticize 
external  forces  and  fail  to  see  how  their 
behavior  can  affect  others.  In  many  in- 
stances, these  students  monopolize  their 
teachers’  time  and  require  constant  instruc- 
tion. 

Students  not  yet  at  level  believe  there  is  only 
one  answer,  and  they  will  not  compromise. 
They  interrupt  the  class  with  negative  or 
incorrect  responses  and  attempt  to  redirect 
the  flow  of  the  class.  These  students  respond 
negatively  toward  the  mistakes  of  others  and 
refuse  to  compromise  or  co-operate.  They 
often  use  slang  terms  to  express  disagree- 
ment. 

In  addition,  students  performing  not  yet  at 
level  seek  to  distract  group  members  from  the 
task  at  hand.  If  they  don’t  frequently  and 
aggressively  monopolize  discussions,  they 
totally  withdraw  from  discussions. 

Participation  skills  of  students  are  an  impor- 
tant part  of  any  program.  With  the  assistance 
of  classroom  teachers,  descriptions  such  as 
those  presented  here  will  continue  to  be 
revised  and  refined. 
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FUTURE  DIRECTIONS 


Broadened  assessment  will  continue  to  be  the 
focus  for  development  and  study  in  the  future. 
The  commitment  of  the  Achievement  Testing 
Unit  is  to  search  for  assessment  methods  that 
will  provide  the  opportunity  for  students  to 
demonstrate  what  they  have  learned  in  other 
areas  of  the  Programs  of  Study  not  normally 
assessed  on  a paper  and  pencil  test. 

In  1992,  broadened  assessment  instruments 
will  be  implemented  as  part  of  the  regular 
achievement  testing  program  and  results  will 
be  reported  as  part  of  the  overall  results. 

Field  tests  will  be  administered  in  other  areas 
as  well: 

Grade  3 

•Participation  Skills  Assessment 

• Science  Performance-based  Assessment 

• Language  Learning  Performance 
Assessment 

Grade  6 

• Listening  and  Viewing  Skills  Assessment 
•Mathematics  Performance-based 

Assessment 

• Science  Performance-based  Assessment 

Grade  9 

•Mathematics  Performance-based 
Assessment 

• Science  Performance-based  Assessment 

• Participation  Skills  Assessment 


For  further  information,  please  contact  a 
member  of  the  Achievement  Testing  Team: 


Greg  Hall — Program  Manager 
Valerie  Doenz — Test  Development  Specialist, 
Social  Studies 

Nola  Aitken — Test  Development  Specialist, 
Mathematics 

Gerard  Lavigne — Test  Development  Special- 
ist, French  Language  Arts 
Jo- Anne  Hug — Test  Development  Specialist, 
English  Language  Arts 
Hank  Boer — Test  Development  Specialist, 
Science 

Achievement  Testing  Program 
Student  Evaluation  Branch 
Alberta  Education 
Devonian  Building,  West  Tower 
11160  Jasper  Avenue 
Edmonton,  Alberta 
T5K  0L2 

Phone:  427-0010 
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APPENDIX  A 
STANDARDS 


The  move  toward  a results-based  curriculum  has  re-emphasized  the  need  for  a clear  delinea- 
tion of  standards  and  of  their  purpose.  All  standards  and  all  methods  of  setting  standards 
require  judgment. 

The  process  of  setting  a standard  can  be  only  as  good  as  the  judgements  that  go  into 
it.  The  standard  will  depend  on  whose  judgements  are  involved  in  the  process.  In 
this  sense,  all  standards  are  subjective.  Yet,  once  a standard  has  been  set,  the 
decisions  based  on  it  can  be  made  objectively.  Instead  of  a separate  set  of  judgements 
for  each  test-taker,  you  will  have  the  same  set  of  judgements  applied  to  all  test- 
takers.  Standards  cannot  be  objectively  determined,  but  they  can  be  objectively 
applied.1 


DEFINITIONS 


The  Achievement  Testing  Program  is  directly 
concerned  with  three  different  but  related 
standards.  These  standards  are  provincial 
curriculum  standards,  provincial  achieve- 
ment standards,  and  provincial  assessment 
standards. 

• Curriculum  Standards  are  the  expected 
student  learnings  sequenced  into  levels. 
They  include  specific  statements  of  knowl- 
edge, skill,  and  attitude  expectations 
against  which  student  performance  is 
judged.  These  standards  are  established  in 
the  process  of  curriculum  development  and 
are  found  in  the  Program  of  Studies  docu- 
ment produced  for  each  course. 

• Achievement  Standards  are  judgments 
that  specify  what  percentage  of  students 
are  expected  to  achieve  an  acceptable  and 
an  excellent  level  of  performance  in  rela- 
tion to  each  course  of  studies,  i.e.,  the 
relevant  curriculum  standards.  It  is  impor- 
tant to  point  out  that  this  judgment  is  not  a 
prediction  of  the  percentage  of  students 
who  will  actually  achieve  acceptable  or 
excellent  levels  of  performance  but  rather 
is  a specification  of  the  percentage  of  stu- 
dents at  a given  age  or  grade  in  school  who 
are  expected  to  perform  at  an  acceptable  or 
excellent  level.  These  standards  apply  to 
school,  jurisdiction,  and  provincial  achieve- 
ment. 


• Assessment  Standards  are  the  criteria 
adopted  forjudging  actual  student  achieve- 
ment relative  to  curriculum  standards. 

They  are  ultimately  expressed  and  applied 
as  test  scores.  They  are  derived  from  an- 
swers to  the  following  or  similar  questions: 
What  scores  must  students  obtain  or 
how  many  questions  and/or  which 
specific  questions  on  a given  test  must 
students  answer  correctly  in  order  for 
their  performance  on  the  test  to  be 
judged  as  acceptable  (or  excellent)? 
These  standards  apply  to  individual  student 
performance. 

Indirectly,  the  Achievement  Testing  Program 
influences  local  targets. 

• Local  Targets  are  the  objectives  set  in 
schools  and  jurisdictions  to  assist  students 
in  moving  toward  or  exceeding  the  provin- 
cial achievement  standards.  These  local 
targets  reflect  the  specific  needs  of  individu- 
als and  groups  within  a specific  community. 

The  Student  Evaluation  Branch  is  responsible 
for  establishing  and  reporting  the  provincial 
assessment  standards  and  shares  responsibil- 
ity with  the  Curriculum  Branch  for  establish- 
ing provincial  achievement  standards  as  they 
relate  to  the  Achievement  Testing  Program. 


lPassing  Scores;  Samuel  A.  Livingston,  Michael  J.  Zieky;  Educational  Testing  Service,  1982 
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PURPOSE 


The  purpose  of  defining  assessment  and 
achievement  standards,  and  the  subsequent 
process  of  setting  the  provincial  assessment 
standard  within  the  sphere  of  the  Achieve- 
ment Testing  Program,  is  to  answer  questions 
such  as: 

• What  is  “Acceptable”  and  “Excellent”  per- 
formance in  relation  to  the  curricular  expec- 
tations for  students  at  the  age  or  grade 
being  tested? 

• What  percentages  of  students  at  the  age  or 
grade  being  tested  ought  to  achieve  an 
“Acceptable”  and  “Excellent”  level,  assum- 
ing adequate  teaching  and  resources? 

• What  scores  on  a specific  test  shall  reason- 
ably represent  “Acceptable”  and  “Excellent” 
performance  respectively? 


• What  are  the  prevailing  strengths  and 
weaknesses  of  Alberta  students  in  relation 
to  the  curriculum  being  tested? 

• What  questions  or  tasks,  when  answered  or 
performed  correctly,  on  a particular  test 
reflect  “Acceptable”  and  “Excellent”  per- 
formance? 

In  essence,  for  each  test  and  level  of  perform- 
ance, the  assessment  standards  setters  are 
challenged  to  answer  the  question,  What 
score  must  a student  obtain  or  how  many 
questions  must  a student  answer  cor- 
rectly to  be  judged  as  having  achieved  an 
acceptable  or  excellent  standard?  Satis- 
factory performance  may  only  be  said  to  exist 
when  the  percentage  of  students  scoring  at  or 
above  the  assessment  standards  is  equal  to  or 
greater  than  the  achievement  standards. 


COMPONENT  STANDARDS  VERSUS  A TOTAL  TEST  STANDARD 


Each  achievement  test  has  traditionally  been 
divided  into  two  major  components.  In 
mathematics,  for  example,  the  components 
are  problem  solving  and  subject  matter.  In  the 
past,  some  students  were  achieving  the  ac- 
ceptable standard  on  the  total  test  while  being 
unable  to  perform  well  on  one  of  the  two  major 
components.  For  example,  a mathematics 
student  might  be  able  to  answer  correctly  all 
or  most  of  the  subject  matter  questions  but  be 
very  weak  on  the  problem-solving  questions. 
Based  on  a "total  test  score,"  this  student 
could  have  been  reported  as  having  achieved 
the  acceptable  standard  in  mathematics, 
whereas  his  or  her  performance  on  one  major 


component  would  not  have  been  acceptable. 
Using  the  Grade  6 Mathematics  results  to 
illustrate,  Figure  A-l  shows  the  percentage  of 
students  achieving  the  acceptable  assessment 
standard  for  each  of  the  two  major  compo- 
nents. Note  that  the  percentage  of  students 
achieving  the  acceptable  standard  for  both 
components  is  less  than  the  percentage 
achieving  the  acceptable  standard  on  each  of 
the  components.  Some  students  achieved  the 
acceptable  standard  on  only  one  of  the  major 
components.  Students,  now  and  in  the  future, 
who  are  judged  to  have  acceptable  perform- 
ance will  have  met  the  standard  on  both 
components. 


Figure  A-l  

Grade  6 Mathematics  Achievement  Test 
Percentage  of  Students  Achieving  Standards  by  Major  Component* 
AT  =31  930(100%) 


16.9% 

Did  not  achieve 
acceptable  standard 
on  either  component 


*Figure  not  to  scale 
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Tables  dealing  with  standards  in  sections  2,  3, 
and  4 of  this  report  show  what  percentage  of 
students  achieved  standards  on  each  of  the 
major  components  and  what  percentage  of 
students  achieved  standards  on  both  compo- 
nents. For  this  year  only,  a total  test  stand- 
ard was  also  determined  and  reported  in 
order  to  provide  for  comparison  with  previous 
results. 


It  should  be  noted  that  standards  were  set 
only  on  the  English  versions  of  the  achieve- 
ment tests  and  they  apply  only  to  students  in 
regular  English  programs.  The  Francophone, 
French  Immersion,  and  other  programs  did 
not  have  sufficient  numbers  to  set  standards 
in  a fair  and  equitable  manner.  Students  in 
these  programs  should  not  be  compared  with 
the  standards  set  for  the  majority  English 
program  population;  however,  local  target 
standards  are  recommended  for  these  groups. 


THE  ASSESSMENT  STANDARD-SETTING  PROCESS 


Figure  A-2  shows  the  model  initiated  by  the 
Student  Evaluation  Branch  to  set  standards 
for  the  1991  achievement  tests.  The  objec- 
tive was  to  widen  the  process  of  setting 
assessment  standards  as  much  as  possi- 
ble over  previous  years  and  especially  to 


provide  for  community  input  and  feed- 
back. The  process  involves  individuals  and 
groups  in  making  judgments  that  contribute 
to  establishing  the  assessment  and  achieve- 
ment standards. 


Figure  A-2 

Process  Model  for  Standard  Setting 
Achievement  Testing  Program 
June  1991 

INPUT 
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Except  for  the  members  of  the  Public  Advisory 
Committee,  the  members  of  all  committees 
outlined  in  Figure  A-2  are  expected  to  be 
highly  knowledgeable  about  both  the  curricu- 
lum and  the  learning  characteristics  of  the 
students  who  are  writing  the  tests.  The  com- 
mittees are  briefly  described  below. 

1.  Curriculum  and  Test  Developers 
Committee 

There  is  one  Curriculum  and  Test  Developers 
Committee  for  each  achievement  test.  The 
committee  consists  of  Alberta  Education 
consultants,  curriculum  developers,  and  test 
development  specialists.  Their  objective  is  to 
recommend  assessment  standards  to  the  Final 
Standards  Review  Committee.  They  may  also 
review  and  make  recommendations  on 
achievement  standards. 


2.  Technical  Review  Committee 

There  is  one  Technical  Review  Committee  for 
each  achievement  test.  Experienced  and 
knowledgeable  educators — teachers,  superin- 
tendents, university  professors,  and  Alberta 
Education  consultants — meet  with  the  devel- 
opers of  each  test.  Their  purpose  is  to  review 
the  test  and  make  recommendations  for 
improvements  where  necessary.  Because  of 
time  constraints,  the  Technical  Review  Com- 
mittee was  not  formally  involved  in  the  set- 
ting of  assessment  standards  for  June  1991. 
However,  in  future  years  this  committee  will 
set  and  recommend  assessment  standards.  In 
addition,  it  will  review  the  appropriateness  of 
current  course  achievement  standards. 


3.  Public  Advisory  Committee 

The  Public  Advisory  Committee,  composed  of 
representatives  from  the  general  public,  met 
and  discussed  standards  for  the  first  time  this 
year.  See  Appendix  B for  more  information  on 
the  proceedings  of  this  committee. 


4.  Teacher  Standards  Committee 

Approximately  20  experienced  teachers  from 
different  areas  of  the  province  are  selected  to 
sit  on  a Teacher  Standards  Committee  for  each 
test.  To  be  selected  for  a committee,  a teacher 
must  have  been  teaching  in  the  grade  and 
subject  area  for  the  previous  two  years.  At 
present,  the  procedures  used  to  aid  teachers  in 
setting  the  assessment  standards  are  the 
modified  Angoff  method  for  Mathematics  and 
Science  subjects  and  the  Nedelsky  method  for 
Humanities  subjects.2  The  teachers  make 
judgments  about  appropriate  assessment 
standards  and  make  recommendations  to  the 
Final  Standards  Review  Committee.  As  well, 
they  review  achievement  standards  for  appro- 
priateness. 

5.  Analytic  Services  Committee 

The  Analytic  Services  Unit  of  the  Student 
Evaluation  Branch  is  the  professional  quality 
control  and  advisory  group  for  the  complete 
process  of  standard  setting.  There  is  a com- 
mittee of  professionals  in  psychometrics  and 
statistics  for  each  test.  A critical  function  is  to 
ensure  that  Alberta  Education’s  standard- 
setting procedures  produce  technically  valid 
results  that  meet  the  strict  requirements  of 
high  quality  professional  studies.  The  commit- 
tee makes  recommendations  for  assessment 
standard-setting  procedures,  leads  individual 
group  discussions,  and  recommends  improve- 
ments where  necessary.  Another  function  is  to 
determine,  independently,  assessment  stand- 
ards through  statistical  analyses  of  current 
student  achievement  data.2 


6.  Final  Standards  Review  Committee 

The  Final  Standards  Review  Committee  con- 
sists of  representatives  from  the  above  com- 
mittees and  is  chaired  by  the  Director  of 
Student  Evaluation.  In  separate  sessions,  the 
recommendations  of  all  test  committees  are 
presented  and,  through  consensus,  final  as- 
sessment standards  are  adopted  for  each  test. 


2A  Consumer’s  Guide  to  Setting  Performance  Standards  on  Criterion-Referenced  Tests,  Ronald  A.  Berk:  Review  of 
Educational  Research,  Spring  1986,  Volume  56. 
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1991  ASSESSMENT  AND  ACHIEVEMENT  STANDARDS 


To  achieve  an  acceptable  standard,  students 
were  required  to  obtain  a certain  score  or  to 
answer  correctly  a certain  number  of  ques- 
tions from  any  part  of  the  test.  To  achieve  a 
standard  of  excellence  in  previous  years, 
students  were  only  required  to  answer  cor- 
rectly a certain  number  of  questions  from  the 
total  test.  In  1991,  for  the  first  time,  students 
were  required  to  satisfy  an  additional  crite- 
rion. To  achieve  a standard  of  excellence  in 
grades  3 and  6,  students  were  required  to 


answer  correctly  a minimum  number  of 
questions  from  the  larger  group  of 
specific  questions  that  were  identified  by 
standard  setters  as  measuring  achievement  at 
an  excellent  level. 

Tables  A-l,  A-2,  and  A-3  show  the  assessment 
and  achievement  standards  adopted  in  June 
1991  by  the  Final  Standards  Review  Commit- 
tee for  the  grades  3,  6,  and  9 achievement 
tests. 


Table  A-l 
Grade  3 Science 

Assessment  and  Achievement  Standards 
June  1991 


Maximum 
Possible 
Raw  Score 

Provincial 
Assessment 
Standard 
(Raw  Score) 

Provincial 
Achievement 
Standard 
(Per  Cent) 

Standard  of  Excellence 

Process  Skills 

36 

32* 

15 

Subject  Matter 

14 

12** 

15 

Both  Major  Components 

36+14 

32+  12 

10 

Acceptable  Standard 

Process  Skills 

36 

19 

85 

Subject  Matter 

14 

8 

85 

Both  Major  Components 

36+14 

19+8 

80 

*The  32  correct  answers  must  include  at  least  nine  of  the  13  questions  that  measure  achievement  at  an  excellent 
level.  The  13  question  numbers  are  5,  6,  13,  16,  22,  35,  37,  38,  39,  44,  45,  48,  and  50.  These  are  13  of  the  36 
Process  Skills  questions  identified  as  questions  that  would  challenge  students  performing  at  a level  of  excellence. 

**The  12  correct  answers  must  include  both  of  the  questions  that  measure  achievement  at  an  excellent  level.  The  two 
question  numbers  are  2 and  17.  These  are  two  of  the  14  Subject  Matter  questions  identified  as  questions  that 
would  challenge  students  performing  at  a level  of  excellence. 
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Table  A-2 

Grade  6 Mathematics 
Assessment  and  Achievement  Standards 
June  1991 


Maximum 
Possible 
Raw  Score 

Provincial 
Assessment 
Standard 
(Raw  Score) 

Provincial 
Achievement 
Standard 
(Per  Cent) 

Standard  of  Excellence 

Problem  Solving 

25 

21* 

15 

Subject  Matter 

30 

27 

15 

Both  Major  Components 

25+  30 

21+27 

10 

Acceptable  Standard 

Problem  Solving 

25 

13 

85 

Subject  Matter 

30 

17 

85 

Both  Major  Components 

25+  30 

13  + 17 

80 

*The  21  correct  answers  must  include  at  least  nine  of  the  11  problem-solving  questions  that  measure  achievement  at 
the  excellent  level.  The  11  question  numbers  are  4,  19,  25,  27,  32,  41,  50,  51,  52,  53,  and  54. 


Table  A-3 

Grade  9 Social  Studies 
Assessment  and  Achievement  Standards 
June  1991 


Maximum 
Possible 
Raw  Score 

Provincial 
Assessment 
Standard 
(Raw  Score) 

Provincial 
Achievement 
Standard 
(Per  Cent) 

Standard  of  Excellence 

Facts,  Concepts,  and 
Generalizations 

33 

29 

15 

Process  Skills 

67 

53 

15 

Both  Major  Components 

33  + 67 

29  + 53 

10 

Acceptable  Standard 

Facts,  Concepts,  and 
Generalizations 

33 

17 

85 

Process  Skills 

67 

31 

85 

Both  Major  Components 

33  + 67 

17  + 31 

80 

During  1991,  much  review  and  discussion  of  standards  was  undertaken  before  adjusting  the 
process  of  standard  setting  to  meet  the  current  and  impending  needs  of  Alberta  Education’s 
Achievement  Testing  Program.  The  exercise  was  enlightening  and  useful.  However,  we  do  not 
expect  that  the  process  will  fit  the  needs  of  all.  We  invite  your  questions,  comments,  and 
recommendations.  Please  send  your  observations  to  Michael  Robinson,  Assistant  Director, 
Analytic  Services,  Student  Evaluation  Branch,  at  427-2948. 
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APPENDIX  B 

PUBLIC  REVIEW  OF  ACHIEVEMENT  STANDARDS 

AND  RESULTS 


On  July  24  and  25, 1991,  representatives  of 
nine  business,  professional,  and  community 
organizations  reviewed  achievement  test 
results  and  discussed  standards.  This  was 
the  first  time  such  a meeting  was  held.  It 
was  planned  to  ensure  that  input  from  groups 
representing  "public"  interests  would  be 
considered  in  the  overall  reporting  of  results. 
The  purpose  of  the  meeting  was  to  gather 
direct  public  input  into  the  question,  How 
well  should  Alberta  students  achieve  in 
Grade  3 Science,  Grade  6 Mathematics, 
and  Grade  9 Social  Studies? 

The  two-day  meeting  was  designed  to  collect 
information  that  would  contribute  to  a better 
understanding  of  how  well  our  students  in 
Alberta  are  doing.  This  was  accomplished  by 
exploring  and  discussing  answers  to  four 
questions.  Each  question  is  listed  below, 
along  with  the  results  of  the  meeting. 

• What  do  members  of  the  public  expect 
students  to  know  and  be  able  to  do  in 
Grade  3 Science,  Grade  6 Mathematics, 
and  Grade  9 Social  Studies? 

Comments:  For  each  subject  area,  partici- 
pants listed  the  knowledge  and  skills  that 
they  felt  were  important  learnings  for  stu- 
dents to  achieve  at  each  grade  level. 

The  following  key  points  highlight  the  group’s 
views: 

Science  3 

Students  should  be  able  to: 

• demonstrate  a curiosity  toward  science 

• make  decisions  and  take  risks  when 
solving  problems 

• use  science  skills  in  real-life  situations 

• communicate  both  orally  and  in  writing 


Mathematics  6 

Students  should  be  able  to: 

• know  how  to  solve  (at  minimum)  two-step 
problems 

• perform  straightforward  numerical  opera- 
tions (addition,  subtraction,  multiplica- 
tion, division) 

• interpret  and  use  information  using 
various  methods  (e.g.,  graphs) 

• estimate  in  different  situations 

• collect  and  record  data 

Social  Studies  9 

Students  should  be  able  to: 

• think  critically  to  support  decision- 
making skills  (interpret  data,  make 
connections,  integrate  ideas,  etc.) 

• present  both  sides  of  an  issue  clearly,  with 
supporting  information 

• understand  the  structure  of  Canadian 
society  and  related  social  issues 

• use  good  communication  skills  (oral  and 
written) 

• use  good  research  skills 

• How  closely  do  the  achievement  tests 

reflect  the  expectations  described  by 

the  group? 

Participants  reviewed  the  achievement  tests 
and  selected  questions  they  felt  students 
should  be  able  to  answer  correctly  to  demon- 
strate an  acceptable  level  of  performance. 
Through  discussion,  the  knowledge  and  skills 
that  students  require  to  answer  these  ques- 
tions successfully  were  listed.  The  knowledge 
and  skills  listed  were  all  very  close  to  the 
original  expectations  held  by  the  group. 
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One  area  the  group  thought  was  important 
but  was  not  addressed  in  any  of  the  achieve- 
ment tests  was  students’  attitudes  toward 
learning.  Concern  was  also  expressed  that 
student  communication  skills  were  not  evalu- 
ated in  the  mathematics  and  science  tests.  At 
minimum,  a written  component  would  be 
needed  to  do  this.  The  group  was  not  unani- 
mous about  whether  written  skills  in  science 
and  mathematics  should  be  evaluated  if  they 
were  already  evaluated  in  language  arts  and 
social  studies. 

The  group  stated  that  the  writing  assignment 
in  social  studies  was  an  excellent  way  to 
evaluate  student  knowledge  and  skills.  The 
assignment  presented  an  issue  and  required 
students  to  state  and  support  a position  of 
their  choice. 


• How  well  did  students  actually  perform 
on  the  achievement  tests? 

Results  showing  how  well  students  performed 
on  important  multiple-choice  items  were 
presented  to  the  group.  As  well,  a first  draft 
of  the  written-response  analysis  for  social 
studies  was  presented. 

In  all  subject  areas,  students  met  some  of  the 
group’s  standards;  in  some  areas  of  learning, 
achievement  was  below  the  group’s  standards. 
Some  of  these  areas  are  listed  below. 

Science  3 

Met  Standards 

Performance  on  the  test  indicated  that  stu- 
dents: 

• know  basic  plant  and  animal  life  in  Al- 
berta 

• are  able  to  make  deductions  that  require 
reasoning 

Below  Standards 

Performance  on  the  test  indicated  that  many 
students  have  difficulty: 

• making  judgments  related  to  cause  and 
effect 


• understanding  interrelationships  in  the 
environment 

• making  practical  applications  and  solving 
problems 

Mathematics  6 
Met  Standards 

Performance  on  the  test  indicated  that  stu- 
dents: 

• can  perform  operations  (multiplying, 
dividing,  adding,  subtracting) 

• can  perform  one-step  problems 

Below  Standards 

Performance  on  the  test  indicated  that  many 
students  have  difficulty: 

• using  information  and  skills  to  solve  two- 
step  problems 

• computing  simple  averages 

Social  Studies  9 
Met  Standards 

Performance  on  the  test  indicated  that  stu- 
dents: 

• understand  points  of  view 

• know  centrally  planned,  mixed,  and 
market  economies  from  a basic  (or 
definitional)  perspective 

Below  Standards 

Performance  on  the  test  indicated  that  many 
students  have  difficulty: 

• understanding  the  role  of  labor,  the 
entrepeneur,  and  the  government  in  the 
different  economies 

• distinguishing  the  interrelationship  of 
social,  economic,  and  political  factors  in 
the  different  economies 

• writing  effectively  when  choosing  and 
supporting  a position 
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• Did  the  group  feel  that  students’  per- 
formance on  the  achievement  tests  was 
good  enough? 

Generally,  the  group  felt  that,  given  their 
expectations  for  both  levels,  student  perform- 
ance was  not  good  enough. 

Several  other  comments  were  made  during 
the  two-day  meeting: 

• All  participants  indicated  they  had  had  an 
adequate  opportunity  to  express  their  views. 

• Participants  viewed  the  activity/discussion 
approach  as  an  excellent  meeting  format. 

• On  many  occasions,  different  participants 
stated  that  expectations  and  standards  have 
to  be  high  to  ensure  our  children  are  reach- 
ing for  high  goals. 


• More  time  to  delve  into  each  subject  would 
have  yielded  more  specific  commentary  on 
expectations/learnings. 

• This  type  of  meeting  should  be  continued  to 
permit  ongoing  input  from  groups  represent- 
ing public  interests. 

Finally,  because  of  the  value  of  this  meeting, 
and  the  desire  to  continue  gathering  public 
input  into  the  standards  and  expectations 
that  exist  for  student  achievement,  a similar 
meeting  will  be  held  in  the  summer  of  1992. 

A more  detailed  report  on  the  1991  meeting 
was  distributed  to  participants  and  is  avail- 
able upon  request.  Please  contact  Marcia 
Wright  at  427-0010  to  obtain  a copy  of  this 
report. 
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APPENDIX  C 

GUIDELINES  TO  INTERPRETATION  OF  RESULTS 


Following  each  administration  of  the  achieve- 
ment tests,  a Provincial  Report  is  prepared. 
This  report  is  a public  document  that  de- 
scribes the  aggregated  results  obtained  by 
those  students  who  wrote  achievement  tests 
in  a given  year.  Provincial  reports  can  be 
used  by  school  board  members,  superintend- 
ents, principals,  and  teachers  as  they  review 
results  for  students  in  their  own  jurisdictions 
and  schools.  By  using  the  Provincial  Report 
in  this  way,  policymakers  and  educators  can 
check  their  perceptions  of  local  achievement 
against  provincewide  standards  and  trends  in 
the  levels  of  achievement. 

This  Provincial  Report  describes  the  re- 
sults achieved  by  students  who  wrote  the 
June  1991  achievement  tests  in  Grade  3 
Science,  Grade  6 Mathematics,  and  Grade  9 
Social  Studies. 

This  year,  for  the  first  time,  jurisdiction 
and  school  reports  were  sent  to  superin- 
tendents and  principals  before  the  Provincial 
Report  was  ready  for  distribution.  Policy- 
makers and  educators  in  each  jurisdiction  are 
encouraged  to  study  carefully  the  provincial 
results  and  their  own  test  results. 


It  should  be  noted  that  results  for  groups  of 
fewer  than  25  students  should  be  interpreted 
with  caution.  Results  for  such  groups  can  be 
unduly  affected  by  the  presence  of  a few 
extreme  scores  when  the  analyses  are  per- 
formed to  produce  the  respective  group  statis- 
tics. Consequently,  any  generalizations  may 
appear  to  indicate  performance  that  is  higher 
or  lower,  better  or  worse,  than  it  is  in  relation 
to  the  criteria  applied. 

Educators  at  the  school  and  jurisdiction  levels 
can  make  three  kinds  of  comparisons  to  decide 
if  the  achievement  of  their  students  is  "good 
enough."  One  comparison  is  in  relation  to 
expectations  or  standards;  another  is  in 
relation  to  the  achievement  of  students  in  the 
entire  province  of  Alberta;  and  the  third  is  to 
look  at  the  jurisdiction  results  in  1987,  when 
tests  in  these  subjects  were  last  administered. 
After  making  these  comparisons,  teachers, 
principals,  and  superintendents  can  identify 
the  strengths  in  the  programs  that  were 
delivered  in  their  grades  3,  6,  and  9 class- 
rooms. Where  the  results  show  weaknesses, 
changes  can  be  made  where  possible  and 
desirable. 


USE  OF  THE  REPORTS 


The  reports  are  not  intended  to  be  used  as  the 
basis  for 

• making  decisions  about  student  placement 
or  promotion 

•evaluating  teacher  performance 
•comparing  performance  between  or  among 
schools. 

Administrators  in  each  jurisdiction  should 
apply  separate  locally  developed  teacher, 
school,  and  school  system  evaluation  policies 
to  the  tasks  of  evaluating  teacher  and  school 
performance. 


The  information  provided  in  the  reports  is 
factual  regarding  what  the  test  results  are. 
The  interpretation  of  this  information — 
hypothesizing  why  results  are  as  they  are — 
involves  consideration  of  the  many  factors  and 
variables  that  contribute  to  achievement. 

In  addition,  it  must  be  noted  that  the  informa- 
tion in  these  reports  is  limited  to  selected 
objectives  of  the  Program  of  Studies.  Many 
important  aspects  of  learning  cannot  be 
measured  by  the  time-limited,  paper  and 
pencil  achievement  tests. 
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Assessment  Standards  and  Achievement  Standards 


Standards  were  confirmed  for  each  achieve- 
ment test  and  are  reported  both  here  in  the 
Provincial  Report  (sections  1,  2,  3,  and  4 and 
Appendix  A)  and  in  tables  2 and  3 of  the 
jurisdiction  and  school  reports.  Criteria 
representing  a standard  of  excellence  and 
an  acceptable  standard  were  confirmed  for 
each  of  the  two  major  reporting  categories 
(components)  of  the  test.  These  criteria  are 
referred  to  as  assessment  standards.  The 
assessment  standards  are  supplemented  by 
achievement  standards,  which  specify  expec- 
tations for  the  percentage  of  students  in  the 
province  expected  to  meet  the  assessment 
standards.  Appendix  A provides  more  detail 
on  both  the  standards  and  the  process  by 
which  they  have  been  confirmed. 

Standards  are  primarily  reported  in  figures 
1-2  to  1-5,  2-1,  2-2,  3-1,  3-2,  4-1,  4-2,  and 
tables  2-3,  2-4,  3-4,  3-5,  4-4  and  4-5  of  the 
Provincial  Report  and  tables  2 and  3 of  each 
school  and  jurisdiction  report.  These  tables 
show  the  number  of  students  writing  the  test 
who  met  or  exceeded  the  provincial  achieve- 
ment standards,  the  number  who  were  ex- 
pected to  meet  or  exceed  the  standards,  and 
the  percentages  who  met  the  standards  on 
each  or  both  of  the  two  major  components  of 
the  test. 


Table  3 of  each  school  and  jurisdiction  report 
gives  the  same  information  for  the  total  test 
rather  than  by  major  component. 

The  school  and  jurisdiction  tables  also  show 
whether  the  number  of  students  in  that  school 
or  jurisdiction  who  achieved  the  standard  is 
significantly  different  from  the  expected 
number.  A 95%  confidence  interval  is  used  for 
these  purposes;  this  criterion  means  that 
differences  are  only  reported  when  there  is  a 
5%  or  smaller  probability  that  a difference  of 
that  size  could  occur  by  chance.  For  schools 
and  jurisdictions  with  fewer  than  five  or  six 
students,  significances  cannot  be  calculated, 
and  the  indications  are  omitted.  Although  the 
statistical  tests  take  the  number  of  students 
into  consideration,  it  is  a useful  rule  of  thumb 
that  results  for  groups  of  fewer  than  25  stu- 
dents must  be  interpreted  with  particular 
caution.  Chance  variation  in  small  groups  is 
greater. 

Educators  interpreting  these  reports  are 
encouraged  to  consider  how  well  their  stu- 
dents have  done  compared  to  the  assessment 
standards  and  how  well  schools  and  jurisdic- 
tions have  done  compared  to  the  achievement 
standards. 


Comparing  Results  to  Average  Scores 


Overall  test  results  are  presented  in  relation 
to  provincial  standards.  Tables  4 and  5 of 
each  jurisdiction  and  school  report  also  pro- 
vide jurisdiction  or  school  average  scores  for 
each  reporting  category  or  subtest.  Each  of 
these  scores  may  be  compared  to  the  provin- 
cial average  for  the  same  reporting  category 
or  subtest  to  determine  if  differences  exist. 
Tables  2-7,  3-8,  and  4-8  of  the  Provincial 
Report  provide  similar  information. 


The  importance  of  differences  that  may  exist 
between  jurisdiction  or  school  averages  and 
provincial  averages  is  not  always  clear.  To 
aid  in  the  interpretation  of  differences  be- 
tween the  averages,  jurisdiction  and  school 
reports  indicate  when  differences  are  signifi- 
cant using  a 95%  confidence  interval. 
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FACTORS  LIMITING  THE  INTERPRETATION  OF  TEST  RESULTS 


Educators  who  are  interpreting  results  must 
take  into  account  the  following  limitations: 

1.  Paper  and  pencil  tests  necessarily  measure 
reading  achievement  in  the  content  area 
being  tested.  Standards  built  into  each 
achievement  test  reflect  the  reading  level 
expectation  for  the  grade  level  tested. 
Jurisdictions  should  consider  the  average 
reading  level  of  their  grades  3,  6,  and  9 
students,  as  reading  levels  below  these 
grades  will  have  an  effect  on  test  results. 


2.  If  more  than  10%  of  eligible  students  in  a 
jurisdiction  did  not  write  a test,  the  re- 
ported statistics  for  that  jurisdiction  may 
not  accurately  represent  the  true  level  of 
achievement. 

3.  Consideration  should  be  given  to  the 
degree  to  which  students  in  particular 
classes  or  grades  were  motivated  to  per- 
form to  their  level  of  ability. 


FACTORS  THAT  MAY  AFFECT  STUDENT  ACHIEVEMENT 


Results  on  the  achievement  tests  may  have 
been  affected  by: 

1.  Environment 

•community  environment 

• school  environment  including  staff  morale 

• socioeconomic  background 
•family  circumstances 


3.  Availability  and  Appropriateness  of 
Resources 
•programs  of  study 
•curriculum  guides 
•resource  materials 
•library  services 
• current  textbooks 
•references 


2.  Personal  Factors 
•ability 
•attitude 

• motivation 

• aspiration 

• academic  background 
•learning  style 

•professional  support 

•teaching  strategies 

•timetabling  constraints  or  influences 


4.  Instruction 

•teacher  qualifications  (i.e.,  to  teach  a 
particular  subject) 

•teacher  experience 
•teacher  morale 
•professional  development 
• staff  turnover 
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A SYSTEMATIC  APPROACH  TO  THE  EFFECTIVE  USE  OF  TEST  RESULTS 


Achievement  test  results  can  be  used  con- 
structively as  one  means  of  improving  the 
quality  of  education.  A systematic  use  of  the 
test  results  would  include  the  following  steps: 

1.  Comparing  test  results  for  a jurisdiction  or 
school  to  the  provincial  results.  Be  sure 
that  your  comparisons  include  the 
•total  test  score, 

•total  and  subtest  scores  for  multiple- 
choice  questions, 

• total  and  subtest  scores  for  written 
response  assignments  (when  appropri- 
ate), 

•individual  multiple-choice  question 
results,  and 


•individual  written-response  question 
results  (when  appropriate). 

2.  Noting  any  patterns,  anomalies,  and/or 
interrelationships  in  the  results. 

3.  Hypothesizing  relationships  between  your 
observations  and  any  of  the  factors  listed 
in  these  guidelines  that  may  have  had  an 
effect  on  achievement  or  achievement  test 
results. 

4.  Considering  and  implementing  a plan  that 
will  help  to  improve  the  quality  of  educa- 
tion for  students. 


AN  ADMINISTRATIVE  MODEL  FOR  THE  EFFECTIVE  USE  OF 
ACHIEVEMENT  TEST  RESULTS 

The  following  model  may  be  useful  for  those  who  wish  to  develop  a constructive  system  for 
interpreting  achievement  test  results. 

Basic  Principles 


1.  It  is  desirable  and  feasible  for  teachers 
and  school  administrators  to  make  use  of 
achievement  test  results  in  analysing  the 
performance  of  their  own  students. 

2.  It  is  more  constructive  for  schools  to 
develop  their  own  analyses,  interpreta- 
tions, and  action  plans  than  to  have  these 
imposed  externally. 

3.  The  impact  of  factors  such  as  those  listed 
in  these  guidelines  should  be  analysed  and 
discussed  when  reviewing  achievement 
test  results. 

4.  Subtest  or  reporting  category  results  are 
usually  more  informative  than  total  test 
scores. 


5.  Generalizations  should  be  stated  with 
caution  and  should  be  supported  by  evi- 
dence that  is  independent  of  achievement 
test  results. 

6.  It  is  neither  desirable  nor  productive  to 
compare  the  results  of  different  schools. 

7.  Achievement  tests  measure  many  of  the 
objectives  specified  by  the  curriculum. 
However,  skills  and  concepts  that  are  not 
measured  by  the  achievement  tests  are 
also  to  be  taught  and  evaluated  at  the 
local  level. 

8.  Staff  discussions  as  well  as  written  reports 
are  useful  means  of  ensuring  that  results 
are  appropriately  interpreted  and  used. 
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Suggested  Content  for  Interpretation  of  Individual  School  Results 


1.  Subject,  grade  level,  and  date  of  achieve- 
ment test  administration 

2.  Number  of  students  who  wrote  the 
achievement  test 

3.  Profiles  of  students  or  groups  who  wrote 
the  achievement  test,  which  include 

— previous  performances 
— number  of  students  repeating  the  grade 

4.  Program  emphases,  such  as  hours  of 
instruction,  skill  and  content  emphases 

5.  Instructional  practices,  such  as  methodol- 
ogy, resources,  and  the  relationship  be- 
tween the  program  offered  and  the  provin- 
cial curriculum 


6.  Program  objectives  not  measured  by  the 
achievement  test 

7.  School  results  compared  with  provincial 
results  on  subtests 

8.  Current  school  results  compared  with 
those  of  previous  administrations 

9.  Discussion  of  item  results,  identification  of 
common  student  errors,  and  suggestions  of 
ways  for  reducing  the  misunderstanding 
that  leads  to  these  errors 

10.  Recommendations  for  the  following  year 
or  semester 

11.  Summary  and  concluding  comments 


Suggested  Reporting  Structure 


1.  Teachers  and/or  the  principal  analyse  the 
results  and  prepare  a written  report  about 
each  administration  of  an  achievement 
test. 

2.  The  principal  reviews  and  signs  the  report. 

3.  The  report  is  shared  with  central  office 
supervisory  personnel. 

4.  The  appropriate  central  office  supervisory 
personnel  prepare  a written  response  to 
the  report,  with  copies  of  the  response 
going  to  the  teachers  and  the  principal. 


5.  If  possible,  all  involved  staff  meet  to 
discuss  the  report  and  the  response. 

6.  Reports  are  used  to  improve  the  program 
and  maximize  future  opportunities  for 
student  success. 

7.  When  large  differences  exist  between 
expected  and  actual  achievement  test 
results  over  time,  consideration  should  be 
given  to  conducting  a formal  program 
evaluation. 
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APPENDIX  D 

REPORTING  TO  PARENTS 


What  are  the  achievement  tests? 

The  achievement  tests  are  provincial  govern- 
ment tests  administered  in  Alberta  schools  to 
grades  3,  6,  and  9 students  in  language  arts, 
social  studies,  science,  and  mathematics. 

What  is  the  purpose  of  the  achievement 
tests? 

The  achievement  tests  help  Alberta  Education 
to  communicate  provincial  expectations  and 
results  for  levels  of  student  performance  in 
language  arts,  social  studies,  science,  and 
mathematics.  The  tests  enable  Alberta  Edu- 
cation to  monitor  the  level  of  achievement  of 
students  throughout  Alberta.  The  results  also 
help  local  school  board,  principals,  and  teach- 
ers identify  the  strengths  and  weaknesses  in 
their  implementation  of  these  subjects. 

How  many  achievement  tests  will  my 
child  have  to  write? 

Students  write  only  one  achievement  test  in 
Grade  3,  one  in  Grade  6,  and  one  in  Grade  9. 
Tests  are  rotated  so  that  a different  subject  is 
tested  each  year.  In  1991,  Grade  3 students 
wrote  the  science  achievement  test,  Grade  6 
students  wrote  the  mathematics  achievement 
test,  and  Grade  9 students  wrote  the  social 
studies  achievement  test.  In  1992,  Grade  3 
students  will  write  the  social  studies  achieve- 
ment test,  Grade  6 students  will  write  the 
language  arts  achievement  test,  and  Grade  9 
students  will  write  the  mathematics  achieve- 
ment test. 

How  should  I prepare  my  child  to  write 
an  achievement  test? 

No  preparation  beyond  normal  classroom 
instruction  is  required  to  write  an  achieve- 
ment test.  While  students  should  be  encour- 
aged to  do  their  best,  a good  night’s  sleep  and 


a relaxed,  confident  approach  to  testing  are 
the  best  possible  preparation. 

How  much  do  these  tests  count  for  my 
child? 

The  achievement  tests  do  NOT  affect  stu- 
dents’ final  marks.  The  classroom  teacher  is 
responsible  for  evaluating  students  and 
awarding  final  marks.  Achievement  test 
results  are  not  released  by  Alberta  Education 
until  October,  long  after  students’  marks  have 
been  determined  by  the  classroom  teacher. 

How  do  achievement  test  results  help 
classroom  teachers? 

Achievement  test  results  provide  feedback  on 
student  achievement  to  school  boards,  princi- 
pals, and  teachers.  For  example,  teachers  in  a 
school  that  consistently  scores  high  on  one 
part  of  the  curriculum  but  low  on  another  may 
wish  to  examine  their  programs  to  see  if 
changes  are  needed  to  achieve  a better  in- 
structional balance. 

What  are  the  limitations  of  the  achieve- 
ment tests? 

Paper  and  pencil  tests  cannot  easily  measure 
such  things  as  laboratory  skills,  small  group 
discussions,  and  creative  thinking.  Thus, 
some  student  strengths  can  be  evaluated  only 
by  the  classroom  teacher.  Also,  a single  test 
cannot  reveal  as  much  about  a student’s 
development  and  growth  as  can  evaluation  by 
the  classroom  teacher  over  the  course  of  a full 
school  year. 

What  advantage  do  achievement  tests 
have  over  other  standardized  tests? 

Unlike  commercially  developed  tests,  achieve- 
ment tests  are  based  specifically  on  Alberta’s 
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programs  of  study  and  are  designed,  written, 
and  evaluated  by  experienced  classroom 
teachers  from  across  the  province.  Tests 
developed  elsewhere  may  not  reflect  curricu- 
lum or  standards  appropriate  for  Alberta. 

How  do  I interpret  achievement  test  re- 
sults? 

The  Achievement  Testing  Program  Provincial 
Report  includes  guidelines  for  interpreting 
results.  Readers  are  cautioned  not  to  overgen- 
eralize conclusions  based  on  a single  admini- 
stration of  the  test.  Results  should  be  com- 
pared with  expectations  or  with  the  results  of 
previous  achievement  tests  in  the  same  sub- 
ject. Any  trends  that  are  observed  in  the 
scores  must  then  be  interpreted  in  the  context 
of  a variety  of  factors  that  could  affect  student 
achievement,  such  as  the  school  and  commu- 
nity environment,  students’  socioeconomic 
background,  and  available  learning  resources. 

Comparisons  between  districts,  schools,  or 
classrooms  are  likely  to  prove  misleading  and 
are  therefore  discouraged. 


Can  I find  out  how  my  child  did  on  the 
achievement  test ? 

Individual  results  for  the  achievement  tests 
are  made  available  to  school  principals.  Since 
the  tests  are  designed  to  gather  information 
on  groups  of  students  and  not  on  individuals, 
individual  results  must  be  interpreted  with 
caution. 

Where  can  I get  additional  information 
about  the  Achievement  Testing  Program ? 

Bulletins  describing  the  content  of  the  coming 
year’s  achievement  tests  and  the  Provincial 
Report  describing  the  results  of  the  previous 
year’s  testing  are  distributed  to  schools  each 
fall.  Requests  for  copies  of  these  publications 
or  questions  and  comments  regarding  the 
Achievement  Testing  Program  should  be 
directed  to: 

Mr.  Dennis  Belyk 
Assistant  Director 

Achievement  Testing  and  Diagnostic 
Evaluation  Programs 
Student  Evaluation  Branch 
Alberta  Education 
Devonian  Building,  West  Tower 
11160  Jasper  Avenue 
Edmonton,  Alberta  T5K  0L2 
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APPENDIX  E 

DEVELOPING  ACHIEVEMENT  TESTS 


The  Student  Evaluation  Branch  develops 
achievement  tests  that  measure  student 
achievement  at  the  grades  3,  6,  and  9 levels. 
Provincewide  testing  in  language  arts,  math- 
ematics, science,  and  social  studies  follows  a 
four-year  cycle  for  each  grade  level  and  sub- 
ject. Many  individuals  and  groups  are  in- 
volved in  the  development  of  each  test:  prac- 
tising classroom  teachers,  school  and  central 
office  administrators,  and  representatives  of 
postsecondary  institutions,  the  Curriculum 
Design  Branch,  the  Language  Services 
Branch,  Regional  Offices,  and  the  Student 
Evaluation  Branch.  Student  Evaluation 
Branch  staff  ensure  that  valid  and  reliable 
tests  are  developed. 

The  following  is  a summary  of  the  phases  of 
the  test  development  process: 

• Planning 

• Approving  Blueprints 

• Developing  Test  Questions 

• Constructing  and  Administering 
Field  Tests 

• Analysing  and  Revising 

• Constructing  Final  Field  Tests 

• Approving  Final  Field  Tests 

• Administering  Final  Field  Tests 

• Constructing  the  Final  Test 

• Preparing  and  Administering  the 
Final  Test 

• Marking 

• Analysing  and  Reporting  the  Results 


Under  normal  circumstances,  it  takes  three 
years  to  complete  all  phases  of  the  process. 


PLANNING 

Test  developers  ensure  that  the  design  of  each 
achievement  test  reflects  the  goals  and  objec- 
tives of  the  Program  of  Studies  and  the  cur- 
riculum specifications  for  each  subject.  Plan- 
ning takes  into  consideration  those  parts  of 
the  program  that  are  testable  in  a paper  and 
pencil  format  within  a given  time  frame. 
Teachers  and  consultants  from  across  the 
province  assist  in  preparing  the  design  of  each 
test. 

Test  developers  prepare  an  interim  test 
blueprint  (an  overall  plan  used  to  guide  the 
development  of  a test).  Questions  that  must 
be  addressed  at  this  point  are: 

•What  knowledge  and  skills  should  students 
be  expected  to  possess? 

•What  types  of  questions  will  constitute  the 
test  (multiple  choice,  short  answer,  or  ex- 
tended written  response)? 

•What  weighting  will  each  part  of  the  test  be 
given? 

• How  long  and  how  demanding  should  the 
test  be? 

•How  should  the  results  of  the  test  be  organ- 
ized for  reporting  purposes? 

In  order  to  ensure  that  each  test  will  produce 
meaningful  and  reliable  results,  test  develop- 
ers incorporate  statistical  as  well  as  curricular 
standards  in  the  test  design.  Statistical 
standards  include  projected  test  means,  range 
of  question  difficulty,  and  the  requirements 
for  reporting.  For  example,  the  ideal  mean  of 
a multiple-choice  test  containing  questions 
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with  four  alternatives  is  62.5%.  This  is  the 
midpoint  between  chance  selection  (25%)  and 
perfection  (100%).  The  range  of  difficulty  of 
multiple-choice  questions  is  expected  to  vary 
from  30%  to  85%  to  ensure  that  students  with 
varying  ability  levels  are  challenged. 

Each  dimension  of  the  curriculum  for  which 
results  are  reported  must  contain  at  least  six 
questions  if  the  results  are  to  be  meaningful. 


APPROVING  BLUEPRINTS 

Blueprint  approval  establishes  the  overall 
design  of  the  test,  the  exact  emphases  given  to 
each  category  for  which  results  are  reported, 
and  the  emphases  given  to  the  different 
cognitive  levels. 

The  interim  blueprint  is  reviewed  by  a com- 
mittee of  Alberta  Education  personnel  that 
represents  the  Curriculum  Design  Branch  (or 
Language  Services  Branch),  Regional  Office 
consultants,  and  the  Student  Evaluation 
Branch.  This  committee  makes  recommenda- 
tions to  the  Director  of  the  Student  Evalua- 
tion Branch. 

The  blueprint  recommended  by  the  Alberta 
Education  committee  is  then  reviewed  by  a 
Test  Review  Committee,  which  consists  of 
members  nominated  by  the  Alberta  Teachers’ 
Association,  the  Conference  of  Alberta  School 
Superintendents,  postsecondary  institutions, 
and  Alberta  Education.  This  committee 
makes  recommendations  to  the  Director  of  the 
Student  Evaluation  Branch. 


DEVELOPING  TEST  QUESTIONS 

Following  blueprint  approval,  committees  of 
practising  classroom  teachers  working  at  the 
appropriate  grade  level  are  formed,  and 
question  development  meetings  are  held. 
These  committees  develop  new  test  questions 
that  reflect  the  goals  and  objectives  of  the 
Program  of  Studies  and  curriculum  specifica- 
tions. Where  necessary,  question  developers 
are  trained  in  the  principles  of  question 
construction.  Questions  built  in  committee 


are  then  screened  for  format,  validity,  blue- 
print ‘fit’,  and  other  design  considerations. 


CONSTRUCTING  AND  ADMINISTERING 
FIELD  TESTS 

After  careful  editing  and  formatting  of  ques- 
tions developed  by  the  teacher  committees, 
field  tests  are  constructed.  Any  required 
artwork  is  completed  during  this  phase  of  the 
test  development  process. 

With  permission  from  school  and  jurisdiction 
personnel,  field  tests  are  sent  to  a number  of 
teachers  throughout  Alberta.  The  students 
involved  are  representative  of  the  student 
population  for  which  the  test  has  been  de- 
signed. A minimum  sample  of  150  students 
writes  each  field  test. 

Teachers  who  administer  a field  test  are 
asked  to  comment  in  writing  on  the  following: 

•reading  level 

•how  closely  the  question  matches  the  way  in 
which  a concept  was  taught 

• level  of  difficulty  of  the  questions 

• quality  of  the  questions  and  graphics 
•errors  of  any  kind 

The  results  from  the  administration  of  this 
initial  round  of  field  tests  are  used  to  validate 
content,  to  determine  difficulty  levels,  and  to 
ensure  that  questions  are  expressed  clearly. 
Special  field  tests  are  also  constructed  to  “try 
out”  new  assessment  strategies  and  tech- 
niques that  may  be  useful  for  future  assess- 
ments. 


ANALYSING  AND  REVISING 

The  results  of  each  field  test  are  then  ana- 
lysed and  scrutinized  to  determine  whether 
individual  questions  require  revision. 
Teacher  comments  regarding  the  way  that 
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test  questions  are  structured  and  the  way  that 
a subject  is  being  taught  are  also  carefully 
considered  and  used  to  guide  revision. 

Questions  deemed  to  require  changes  are 
revised  and  submitted  for  further  field  test- 
ing. 

CONSTRUCTING  FINAL  FIELD  TESTS 

Once  the  initial  field  test  results  are  thor- 
oughly analysed  and  questions  requiring 
changes  are  revised,  final  field  tests  are 
constructed.  These  field  tests  follow  the 
approved  blueprint  and  parallel  the  actual 
achievement  test  in  format  and  design. 

Final  field  tests,  like  all  field  tests,  are  sub- 
mitted for  further  validity  checking,  editing, 
and  proofreading.  In  grades  6 and  9,  separate 
tests  in  English  and  in  French  are  developed 
for  language  arts.  At  this  point,  all  other  tests 
for  Grade  6 and  Grade  9 are  translated  into 
French. 


APPROVING  FINAL  FIELD  TESTS 

After  the  final  field  tests  have  been  con- 
structed, a second  meeting  of  the  Alberta 
Education  Committee  that  represents  the 
Curriculum  Design  Branch  (or  Language 
Services  Branch),  Regional  Office  consultants, 
and  the  Student  Evaluation  Branch  is  con- 
vened. This  committee  reviews  the  final  field 
tests  and  makes  recommendations  for  im- 
provement. 

The  Test  Review  Committee,  which  approved 
the  blueprint  in  phase  two  of  the  test  develop- 
ment process,  meets  a second  time  to  review 
and  recommend  for  approval  the  final  field 
tests  and  the  instructions  for  administering 
the  tests.  If  a test  includes  short-answer  or 
extended-writing  questions,  the  Test  Review 
Committee  discusses  standards  of  achieve- 
ment and  marking  standards  appropriate  for 
the  test.  Again,  this  committee  makes  recom- 
mendations to  the  Director  of  the  Student 
Evaluation  Branch. 


ADMINISTERING  FINAL  FIELD  TESTS 

The  final  field  tests  are  administered  and  the 
results  are  used  as  a final  screen  in  selecting 
questions  for  placement  on  the  provincial 
achievement  test.  A minimum  sample  of  250 
students  writes  each  final  field  test.  The 
sample  is  selected  to  include: 

• only  students  who  have  received  instruction 
in  the  course 

• students  representing  a normal  distribution 
of  ability  levels 

• students  from  rural  and  urban  schools 

• students  from  large  and  small  schools 


CONSTRUCTING  THE  FINAL  TEST 

The  construction  of  the  final  test  form  is  based 
upon  information  collected  from  the  final  field 
test  administration.  The  Test  Review  Com- 
mittee is  reconvened  to  review  the  final  test 
form. 

The  test  is  submitted  for  final  validity  check- 
ing, editing,  and  proofreading.  Grade  6 and 
Grade  9 achievement  tests,  in  subjects  other 
than  language  arts,  are  translated  into 
French. 

For  each  test,  an  information  bulletin  is 
prepared  that  outlines  the  design  and  nature 
of  the  upcoming  tests.  These  bulletins  are 
distributed  to  each  school  in  September  to 
facilitate  program  and  instructional  planning 
by  teachers  and  administrators. 


PREPARING  AND  ADMINISTERING  THE 
FINAL  TEST 

The  completed  achievement  test  is  commer- 
cially printed  and  prepared  for  distribution. 
Sufficient  copies  of  the  test  are  mailed  to  each 
school.  Quantities  are  based  on  the  number  of 
students  enrolled  in  the  subject  as  reported  to 
the  Student  Evaluation  Branch.  The  test  is 
administered  to  students  by  their  classroom 
teachers. 
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MARKING 


ANALYSING  AND  REPORTING  THE  RESULTS 


All  written-response  sections  of  the  tests  are 
marked  by  classroom  teachers.  These  teach- 
ers, who  are  recommended  by  their  superin- 
tendents, are  currently  teaching  the  course 
being  evaluated,  have  taught  the  course  for  a 
minimum  of  two  years,  and  hold  a valid 
Alberta  Permanent  Professional  Certificate. 
Student  Evaluation  Branch  staff  train  and 
supervise  the  teachers  during  the  marking 
sessions.  All  multiple-choice  responses  are 
machine  scored. 


A results  report  is  prepared  and  distributed  to 
superintendents,  school  principals,  Alberta 
Education  officials,  and  other  Departments  of 
Education.  This  report  is  also  made  available 
to  the  general  public.  In  addition  to  the 
Achievement  Testing  Program  Provincial 
Report , each  school  and  jurisdiction  receives  a 
statistical  summary  for  its  student  population. 

For  further  information,  please  refer  to  the 
Achievement  Test  Bulletins,  or  call  the  Assist- 
ant Director,  Achievement  Testing  and  Diag- 
nostic Evaluation  Programs,  at  427-0010. 
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ACHIEVEMENT  TESTING  PROGRAM 
PROVINCIAL  REPORT  QUESTIONNAIRE 


The  Student  Evaluation  Branch  strives  to 
produce  documents  that  will  be  useful  to  the 
educational  community.  The  purpose  of  the 
following  questionnaire  is  to  collect  your 
opinions  about  the  Provincial  Report.  These 
opinions  will  be  considered  when  the  content 
and  format  of  the  report  are  reviewed  before 
June  1992. 


Please  take  a moment  to  respond  to  the  fol- 
lowing questions.  Then  detach  this  sheet  and 
send  it  to: 

Mr.  Michael  Robinson 

Assistant  Director,  Analytic  Services 

Student  Evaluation  Branch 

Alberta  Education 

11160  Jasper  Avenue 

Edmonton,  Alberta  T5K  0L2 


USE  OF  THE  REPORT 

Please  check  the  boxes  that  apply  to  you. 

1 . My  present  role  is  primarily  that  of 

• teacher  | 1 

• school  administrator  1 1 

• central  office  administrator  1 1 

• school  board  member  1 1 

• other  (please  specify)  — 

2.  I read  the  report,  but  I DID  NOT  use  it  to  interpret  the  results  attained  by  my  students. 

3.  I read  the  report,  and  I used  it  to  interpret  the  results  attained  by  the  students  in 

• my  classroon  • my  school  • my  jurisdiction 


Please  respond  to  the  following  statement  if  you  have  checked  one  of  the  three  boxes  in  question  3. 

4.  I have  made  use  of  the  results  to  alter  the  educational  program  offered  in 

• my  classroon  • my  school  • my  jurisdiction 


Continued 
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CONTENT  OF  THE  REPORT 


1 . Please  judge  the  usefulness  of  the  information  included  in  the  various  sections  of  the  report  by  checking  the 


appropriate  boxes  below. 

Very 

Adequate 

Of  Some 

Of  No 

Section  1:  Summary  of  Achievement  Test  Results 

Useful 

□ 

for  Use 

□ 

[ 

Use 

=1 

L 

Use 

Section  2:  Grade  3 Science 

1 — 1 

□ 

[ 

mi 

[ 

Section  3:  Grade  6 Mathematics 

CZI 

□ 

[ 

— i 

[ 

Section  4:  Grade  9 Social  Studies 

□ 

□ 

L 

=□ 

[ 

Section  5:  Achievement  By  Gender 

1 1 

□ 

L 

mu 

[ 

Section  6:  Achievement  By  Age 

□ 

□ 

C 

HI 

L 

Section  7:  Achievement  by  Grade  Level 

1 1 

□ 

[ 

mu 

[ 

Section  8:  Achievement  Over  Time 

□ 

1 1 

[ 

=□ 

[ 

Section  9:  Directions:  Broadened  Assessment 

□ 

□ 

[ 

=□ 

L 

2.  If  you  wish,  please  comment  further  on  the  content  of  the  report  in  the  space  below. 


FORMAT  OF  THE  REPORT 

1 . Please  judge  the  usefulness  of  the  report’s  format  by  checking  the  appropriate  boxes  below. 


Very 

Adequate 

Of  Some 

Of  No 

Organization  into  Separate 
Sections 

Useful 

for  Use 

Use 

Use 

I_J 

1 1 

1 1 

1 1 

Double-Column  Presentation 
of  Text 


Presentation  of  Figures 


Presentation  of  Tables 


Blending  of  Information 
in  Text,  Figures,  and  Tables 


2.  If  you  wish,  please  comment  further  on  the  format  of  the  report  in  the  space  below. 


- 102- 


LB  305M  C2  03152  1991 
PROVINCIAL  REPORT  ACHIEVEMENT 
TESTING  PROGRAM  « 


SERIAL 


Ml  3999089G  EOUC 


B49310 


